SGI Freeware 1999 August/SGI Freeware 1999 August.iso/dist/fw_flex.idb/usr/freeware/catman/u_man/cat1/flex.Z/flex

Text File | 1998-05-21 | 155.5 KB | 3,697 lines

     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



     NNNNAAAAMMMMEEEE
      flex - fast lexical analyzer generator

     SSSSYYYYNNNNOOOOPPPPSSSSIIIISSSS
      fffflllleeeexxxx [[[[----bbbbccccddddffffhhhhiiiillllnnnnppppssssttttvvvvwwwwBBBBFFFFIIIILLLLTTTTVVVV77778888++++???? ----CCCC[[[[aaaaeeeeffffFFFFmmmmrrrr]]]] ----oooooooouuuuttttppppuuuutttt ----PPPPpppprrrreeeeffffiiiixxxx
      ----SSSSsssskkkkeeeelllleeeettttoooonnnn]]]] [[[[--------hhhheeeellllpppp --------vvvveeeerrrrssssiiiioooonnnn]]]] [_f_i_l_e_n_a_m_e ...]

     OOOOVVVVEEEERRRRVVVVIIIIEEEEWWWW
      This manual describes    _f_l_e_x, a    tool for generating programs
      that perform pattern-matching    on text.  The manual includes
      both tutorial    and reference sections:

          Description
          a brief overview of the tool

          Some Simple Examples

          Format Of    The Input File

          Patterns
          the extended regular expressions used    by flex

          How The Input Is Matched
          the rules for    determining what has been matched

          Actions
          how to specify what to do when a pattern is matched

          The Generated Scanner
          details regarding the    scanner    that flex produces;
          how to control the input source

          Start Conditions
          introducing context into your    scanners, and
          managing "mini-scanners"

          Multiple Input Buffers
          how to manipulate multiple input sources; how    to
          scan from strings instead of files

          End-of-file Rules
          special rules    for matching the end of    the input

          Miscellaneous Macros
          a summary of macros available    to the actions

          Values Available To The User
          a summary of values available    to the actions

          Interfacing With Yacc
          connecting flex scanners together with yacc parsers




     Page 1                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          Options
          flex command-line options, and the "%option"
          directive

          Performance Considerations
          how to make your scanner go as fast as possible

          Generating C++ Scanners
          the (experimental) facility for generating C++
          scanner classes

          Incompatibilities    With Lex And POSIX
          how flex differs from    AT&T lex and the POSIX lex
          standard

          Diagnostics
          those    error messages produced    by flex    (or scanners
          it generates)    whose meanings might not be apparent

          Files
          files    used by    flex

          Deficiencies / Bugs
          known    problems with flex

          See Also
          other    documentation, related tools

          Author
          includes contact information


     DDDDEEEESSSSCCCCRRRRIIIIPPPPTTTTIIIIOOOONNNN
      _f_l_e_x is a tool for generating    _s_c_a_n_n_e_r_s: programs which
      recognized lexical patterns in text.    _f_l_e_x reads the given
      input    files, or its standard input if    no file    names are
      given, for a description of a    scanner    to generate.  The
      description is in the    form of    pairs of regular expressions
      and C    code, called _r_u_l_e_s. _f_l_e_x generates as output a C
      source file, lllleeeexxxx....yyyyyyyy....cccc,,,, which defines a routine yyyyyyyylllleeeexxxx(((()))).... This
      file is compiled and linked with the ----llllffffllll library to produce
      an executable.  When the executable is run, it analyzes its
      input    for occurrences    of the regular expressions.  Whenever
      it finds one,    it executes the    corresponding C    code.

     SSSSOOOOMMMMEEEE SSSSIIIIMMMMPPPPLLLLEEEE EEEEXXXXAAAAMMMMPPPPLLLLEEEESSSS
      First    some simple examples to    get the    flavor of how one uses
      _f_l_e_x.    The following _f_l_e_x input specifies a scanner which
      whenever it encounters the string "username" will replace it
      with the user's login    name:

          %%



     Page 2                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          username      printf( "%s",    getlogin() );

      By default, any text not matched by a    _f_l_e_x scanner is    copied
      to the output, so the    net effect of this scanner is to copy
      its input file to its    output with each occurrence of
      "username" expanded.    In this    input, there is    just one rule.
      "username" is    the _p_a_t_t_e_r_n and    the "printf" is    the _a_c_t_i_o_n.
      The "%%" marks the beginning of the rules.

      Here's another simple    example:

              int num_lines = 0, num_chars = 0;

          %%
          \n      ++num_lines; ++num_chars;
          .          ++num_chars;

          %%
          main()
              {
              yylex();
              printf( "# of lines = %d,    # of chars = %d\n",
                  num_lines, num_chars );
              }

      This scanner counts the number of characters and the number
      of lines in its input    (it produces no    output other than the
      final    report on the counts).    The first line declares    two
      globals, "num_lines" and "num_chars",    which are accessible
      both inside yyyyyyyylllleeeexxxx(((()))) and in the mmmmaaaaiiiinnnn(((())))    routine    declared after
      the second "%%".  There are two rules, one which matches a
      newline ("\n") and increments    both the line count and    the
      character count, and one which matches any character other
      than a newline (indicated by the "." regular expression).

      A somewhat more complicated example:

          /* scanner for a toy Pascal-like language    */

          %{
          /* need this for the call    to atof() below    */
          #include <math.h>
          %}

          DIGIT    [0-9]
          ID       [a-z][a-z0-9]*

          %%

          {DIGIT}+      {
              printf( "An integer: %s (%d)\n", yytext,
                  atoi(    yytext ) );



     Page 3                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



              }

          {DIGIT}+"."{DIGIT}*     {
              printf( "A float: %s (%g)\n",    yytext,
                  atof(    yytext ) );
              }

          if|then|begin|end|procedure|function      {
              printf( "A keyword: %s\n", yytext );
              }

          {ID}      printf( "An identifier: %s\n", yytext    );

          "+"|"-"|"*"|"/"    printf(    "An operator: %s\n", yytext );

          "{"[^}\n]*"}"    /* eat up one-line comments */

          [    \t\n]+        /* eat up whitespace */

          .          printf( "Unrecognized    character: %s\n", yytext );

          %%

          main( argc, argv )
          int argc;
          char **argv;
          {
          ++argv, --argc;  /* skip over    program    name */
          if ( argc > 0    )
              yyin = fopen(    argv[0], "r" );
          else
              yyin = stdin;

          yylex();
          }

      This is the beginnings of a simple scanner for a language
      like Pascal.    It identifies different    types of _t_o_k_e_n_s    and
      reports on what it has seen.

      The details of this example will be explained    in the
      following sections.

     FFFFOOOORRRRMMMMAAAATTTT OOOOFFFF TTTTHHHHEEEE IIIINNNNPPPPUUUUTTTT FFFFIIIILLLLEEEE
      The _f_l_e_x input file consists of three    sections, separated by
      a line with just %%%%%%%% in it:

          definitions
          %%
          rules
          %%
          user code



     Page 4                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      The _d_e_f_i_n_i_t_i_o_n_s section contains declarations    of simple _n_a_m_e
      definitions to simplify the scanner specification, and
      declarations of _s_t_a_r_t    _c_o_n_d_i_t_i_o_n_s, which are explained    in a
      later    section.

      Name definitions have    the form:

          name definition

      The "name" is    a word beginning with a    letter or an
      underscore ('_') followed by zero or more letters, digits,
      '_', or '-' (dash).  The definition is taken to begin    at the
      first    non-white-space    character following the    name and
      continuing to    the end    of the line.  The definition can
      subsequently be referred to using "{name}", which will
      expand to "(definition)".  For example,

          DIGIT    [0-9]
          ID       [a-z][a-z0-9]*

      defines "DIGIT" to be    a regular expression which matches a
      single digit,    and "ID" to be a regular expression which
      matches a letter followed by zero-or-more letters-or-digits.
      A subsequent reference to

          {DIGIT}+"."{DIGIT}*

      is identical to

          ([0-9])+"."([0-9])*

      and matches one-or-more digits followed by a '.' followed by
      zero-or-more digits.

      The _r_u_l_e_s section of the _f_l_e_x    input contains a series    of
      rules    of the form:

          pattern    action

      where    the pattern must be unindented and the action must
      begin    on the same line.

      See below for    a further description of patterns and actions.

      Finally, the user code section is simply copied to lllleeeexxxx....yyyyyyyy....cccc
      verbatim.  It    is used    for companion routines which call or
      are called by    the scanner.  The presence of this section is
      optional; if it is missing, the second %%%%%%%% in the input file
      may be skipped, too.

      In the definitions and rules sections, any _i_n_d_e_n_t_e_d text or
      text enclosed    in %%%%{{{{ and %%%%}}}} is    copied verbatim    to the output



     Page 5                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      (with    the %{}'s removed).  The %{}'s must appear unindented
      on lines by themselves.

      In the rules section,    any indented or    %{} text appearing
      before the first rule    may be used to declare variables which
      are local to the scanning routine and    (after the
      declarations)    code which is to be executed whenever the
      scanning routine is entered.    Other indented or %{} text in
      the rule section is still copied to the output, but its
      meaning is not well-defined and it may well cause compile-
      time errors (this feature is present for _P_O_S_I_X compliance;
      see below for    other such features).

      In the definitions section (but not in the rules section),
      an unindented    comment    (i.e., a line beginning    with "/*") is
      also copied verbatim to the output up    to the next "*/".

     PPPPAAAATTTTTTTTEEEERRRRNNNNSSSS
      The patterns in the input are    written    using an extended set
      of regular expressions.  These are:

          x         match the character 'x'
          .         any character (byte) except newline
          [xyz]     a "character class"; in this case, the    pattern
               matches either an 'x', a 'y', or a 'z'
          [abj-oZ]     a "character class" with a range in it; matches
               an 'a', a 'b', any letter from 'j' through 'o',
               or a    'Z'
          [^A-Z]     a "negated character class", i.e., any    character
               but those in    the class.  In this case, any
               character EXCEPT an uppercase letter.
          [^A-Z\n]     any character EXCEPT an uppercase letter or
               a newline
          r*     zero or more r's, where r is any regular expression
          r+     one or    more r's
          r?     zero or one r's (that is, "an optional    r")
          r{2,5}     anywhere from two to five r's
          r{2,}     two or    more r's
          r{4}     exactly 4 r's
          {name}     the expansion of the "name" definition
             (see above)
          "[xyz]\"foo"
             the literal string: [xyz]"foo
          \X     if X is an 'a', 'b', 'f', 'n',    'r', 't', or 'v',
               then    the ANSI-C interpretation of \x.
               Otherwise, a    literal    'X' (used to escape
               operators such as '*')
          \0     a NUL character (ASCII    code 0)
          \123     the character with octal value    123
          \x2a     the character with hexadecimal    value 2a
          (r)     match an r; parentheses are used to override
               precedence (see below)



     Page 6                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          rs     the regular expression    r followed by the
               regular expression s; called    "concatenation"


          r|s     either    an r or    an s


          r/s     an r but only if it is    followed by an s.  The
               text    matched    by s is    included when determining
               whether this    rule is    the "longest match",
               but is then returned    to the input before
               the action is executed.  So the action only
               sees    the text matched by r.    This type
               of pattern is called    trailing context".
               (There are some combinations    of r/s that flex
               cannot match    correctly; see notes in    the
               Deficiencies    / Bugs section below regarding
               "dangerous trailing context".)
          ^r     an r, but only    at the beginning of a line (i.e.,
               which just starting to scan,    or right after a
               newline has been scanned).
          r$     an r, but only    at the end of a    line (i.e., just
               before a newline).  Equivalent to "r/\n".

             Note that flex's notion of "newline" is exactly
             whatever the C    compiler used to compile flex
             interprets '\n' as; in    particular, on some DOS
             systems you must either filter    out \r's in the
             input yourself, or explicitly use r/\r\n for "r$".


          <s>r     an r, but only    in start condition s (see
               below for discussion    of start conditions)
          <s1,s2,s3>r
             same, but in any of start conditions s1,
               s2, or s3
          <*>r     an r in any start condition, even an exclusive    one.


          <<EOF>>     an end-of-file
          <s1,s2><<EOF>>
             an end-of-file    when in    start condition    s1 or s2

      Note that inside of a    character class, all regular
      expression operators lose their special meaning except
      escape ('\') and the character class operators, '-', ']',
      and, at the beginning    of the class, '^'.

      The regular expressions listed above are grouped according
      to precedence, from highest precedence at the    top to lowest
      at the bottom.  Those    grouped    together have equal
      precedence.  For example,



     Page 7                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          foo|bar*

      is the same as

          (foo)|(ba(r*))

      since    the '*'    operator has higher precedence than
      concatenation, and concatenation higher than alternation
      ('|').  This pattern therefore matches _e_i_t_h_e_r    the string
      "foo"    _o_r the string "ba" followed by zero-or-more r's.  To
      match    "foo" or zero-or-more "bar"'s, use:

          foo|(bar)*

      and to match zero-or-more "foo"'s-or-"bar"'s:

          (foo|bar)*


      In addition to characters and    ranges of characters,
      character classes can    also contain character class
      _e_x_p_r_e_s_s_i_o_n_s. These are expressions enclosed inside [[[[::::    and ::::]]]]
      delimiters (which themselves must appear between the '[' and
      ']' of the character class; other elements may occur inside
      the character    class, too).  The valid    expressions are:

          [:alnum:]    [:alpha:] [:blank:]
          [:cntrl:]    [:digit:] [:graph:]
          [:lower:]    [:print:] [:punct:]
          [:space:]    [:upper:] [:xdigit:]

      These    expressions all    designate a set    of characters
      equivalent to    the corresponding standard C iiiissssXXXXXXXXXXXX function.
      For example, [[[[::::aaaallllnnnnuuuummmm::::]]]] designates those characters for which
      iiiissssaaaallllnnnnuuuummmm(((()))) returns true - i.e., any alphabetic    or numeric.
      Some systems don't provide iiiissssbbbbllllaaaannnnkkkk(((()))),,,,    so flex    defines
      [[[[::::bbbbllllaaaannnnkkkk::::]]]] as a blank or a tab.

      For example, the following character classes are all
      equivalent:

          [[:alnum:]]
          [[:alpha:][:digit:]
          [[:alpha:]0-9]
          [a-zA-Z0-9]

      If your scanner is case-insensitive (the ----iiii flag), then
      [[[[::::uuuuppppppppeeeerrrr::::]]]] and    [[[[::::lllloooowwwweeeerrrr::::]]]] are equivalent to [[[[::::aaaallllpppphhhhaaaa::::]]]]....

      Some notes on    patterns:

      -    A negated character class such as the example "[^A-Z]"



     Page 8                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           above _w_i_l_l _m_a_t_c_h    _a _n_e_w_l_i_n_e unless "\n" (or an
           equivalent escape sequence) is one of the characters
           explicitly present in the negated character class
           (e.g., "[^A-Z\n]").  This is unlike how many other
           regular expression tools    treat negated character
           classes,    but unfortunately the inconsistency is
           historically entrenched.     Matching newlines means that
           a pattern like [^"]* can    match the entire input unless
           there's another quote in    the input.

      -    A rule can have at most one instance of trailing
           context (the '/'    operator or the    '$' operator).    The
           start condition,    '^', and "<<EOF>>" patterns can    only
           occur at    the beginning of a pattern, and, as well as
           with '/'    and '$', cannot    be grouped inside parentheses.
           A '^' which does    not occur at the beginning of a    rule
           or a '$'    which does not occur at    the end    of a rule
           loses its special properties and    is treated as a    normal
           character.

           The following are illegal:

           foo/bar$
           <sc1>foo<sc2>bar

           Note that the first of these, can be written
           "foo/bar\n".

           The following will result in '$'    or '^' being treated
           as a normal character:

           foo|(bar$)
           foo|^bar

           If what's wanted    is a "foo" or a    bar-followed-by-a-
           newline,    the following could be used (the special '|'
           action is explained below):

           foo        |
           bar$        /* action goes here    */

           A similar trick will work for matching a    foo or a bar-
           at-the-beginning-of-a-line.

     HHHHOOOOWWWW TTTTHHHHEEEE IIIINNNNPPPPUUUUTTTT IIIISSSS MMMMAAAATTTTCCCCHHHHEEEEDDDD
      When the generated scanner is    run, it    analyzes its input
      looking for strings which match any of its patterns.    If it
      finds    more than one match, it    takes the one matching the
      most text (for trailing context rules, this includes the
      length of the    trailing part, even though it will then    be
      returned to the input).  If it finds two or more matches of
      the same length, the rule listed first in the    _f_l_e_x input



     Page 9                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      file is chosen.

      Once the match is determined,    the text corresponding to the
      match    (called    the _t_o_k_e_n) is made available in    the global
      character pointer yyyyyyyytttteeeexxxxtttt,,,, and    its length in the global
      integer yyyyyyyylllleeeennnngggg.... The _a_c_t_i_o_n corresponding to the matched
      pattern is then executed (a more detailed description    of
      actions follows), and    then the remaining input is scanned
      for another match.

      If no    match is found,    then the _d_e_f_a_u_l_t _r_u_l_e is executed: the
      next character in the    input is considered matched and    copied
      to the standard output.  Thus, the simplest legal _f_l_e_x input
      is:

          %%

      which    generates a scanner that simply    copies its input (one
      character at a time) to its output.

      Note that yyyyyyyytttteeeexxxxtttt can be defined in two different ways:
      either as a character    _p_o_i_n_t_e_r    or as a    character _a_r_r_a_y. You
      can control which definition _f_l_e_x uses by including one of
      the special directives %%%%ppppooooiiiinnnntttteeeerrrr or %%%%aaaarrrrrrrraaaayyyy in the first
      (definitions)    section    of your    flex input.  The default is
      %%%%ppppooooiiiinnnntttteeeerrrr,,,, unless you use the ----llll lex compatibility option, in
      which    case yyyyyyyytttteeeexxxxtttt will be an array.  The advantage of    using
      %%%%ppppooooiiiinnnntttteeeerrrr is substantially faster scanning and    no buffer
      overflow when    matching very large tokens (unless you run out
      of dynamic memory).  The disadvantage    is that    you are
      restricted in    how your actions can modify yyyyyyyytttteeeexxxxtttt (see    the
      next section), and calls to the uuuunnnnppppuuuutttt(((()))) function destroys
      the present contents of yyyyyyyytttteeeexxxxtttt,,,, which    can be a considerable
      porting headache when    moving between different _l_e_x versions.

      The advantage    of %%%%aaaarrrrrrrraaaayyyy is that you can then modify yyyyyyyytttteeeexxxxtttt
      to your heart's content, and calls to    uuuunnnnppppuuuutttt(((())))    do not destroy
      yyyyyyyytttteeeexxxxtttt (see below).  Furthermore, existing _l_e_x programs
      sometimes access yyyyyyyytttteeeexxxxtttt externally using declarations    of the
      form:
          extern char yytext[];
      This definition is erroneous when used with %%%%ppppooooiiiinnnntttteeeerrrr,,,,    but
      correct for %%%%aaaarrrrrrrraaaayyyy....

      %%%%aaaarrrrrrrraaaayyyy defines yyyyyyyytttteeeexxxxtttt    to be an array of YYYYYYYYLLLLMMMMAAAAXXXX characters,
      which    defaults to a fairly large value.  You can change the
      size by simply #define'ing YYYYYYYYLLLLMMMMAAAAXXXX to a different value in
      the first section of your _f_l_e_x input.     As mentioned above,
      with %%%%ppppooooiiiinnnntttteeeerrrr    yytext grows dynamically to accommodate    large
      tokens.  While this means your %%%%ppppooooiiiinnnntttteeeerrrr scanner can
      accommodate very large tokens    (such as matching entire
      blocks of comments), bear in mind that each time the scanner



     Page 10                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      must resize yyyyyyyytttteeeexxxxtttt it    also must rescan the entire token from
      the beginning, so matching such tokens can prove slow.
      yyyyyyyytttteeeexxxxtttt presently does    _n_o_t dynamically    grow if    a call to
      uuuunnnnppppuuuutttt(((()))) results in too much text being pushed    back; instead,
      a run-time error results.

      Also note that you cannot use    %%%%aaaarrrrrrrraaaayyyy with C++    scanner
      classes (the cccc++++++++ option; see below).

     AAAACCCCTTTTIIIIOOOONNNNSSSS
      Each pattern in a rule has a corresponding action, which can
      be any arbitrary C statement.     The pattern ends at the first
      non-escaped whitespace character; the    remainder of the line
      is its action.  If the action    is empty, then when the
      pattern is matched the input token is    simply discarded.  For
      example, here    is the specification for a program which
      deletes all occurrences of "zap me" from its input:

          %%
          "zap me"

      (It will copy    all other characters in    the input to the
      output since they will be matched by the default rule.)

      Here is a program which compresses multiple blanks and tabs
      down to a single blank, and throws away whitespace found at
      the end of a line:

          %%
          [    \t]+        putchar( ' ' );
          [    \t]+$        /* ignore this token */


      If the action    contains a '{',    then the action    spans till the
      balancing '}'    is found, and the action may cross multiple
      lines.  _f_l_e_x knows about C strings and comments and won't be
      fooled by braces found within    them, but also allows actions
      to begin with    %%%%{{{{ and will consider the action    to be all the
      text up to the next %%%%}}}} (regardless of    ordinary braces    inside
      the action).

      An action consisting solely of a vertical bar    ('|') means
      "same    as the action for the next rule."  See below for an
      illustration.

      Actions can include arbitrary    C code,    including rrrreeeettttuuuurrrrnnnn
      statements to    return a value to whatever routine called
      yyyyyyyylllleeeexxxx(((()))).... Each    time yyyyyyyylllleeeexxxx(((()))) is    called it continues processing
      tokens from where it last left off until it either reaches
      the end of the file or executes a return.

      Actions are free to modify yyyyyyyytttteeeexxxxtttt except for lengthening it



     Page 11                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      (adding characters to    its end--these will overwrite later
      characters in    the input stream).  This however does not
      apply    when using %%%%aaaarrrrrrrraaaayyyy (see above); in that case, yyyyyyyytttteeeexxxxtttt
      may be freely    modified in any    way.

      Actions are free to modify yyyyyyyylllleeeennnngggg except they    should not do
      so if    the action also    includes use of    yyyyyyyymmmmoooorrrreeee(((()))) (see below).

      There    are a number of    special    directives which can be
      included within an action:

      -    EEEECCCCHHHHOOOO copies yytext to the scanner's output.

      -    BBBBEEEEGGGGIIIINNNN followed by the name of a start condition places
           the scanner in the corresponding    start condition    (see
           below).

      -    RRRREEEEJJJJEEEECCCCTTTT directs the scanner to proceed on    to the "second
           best" rule which    matched    the input (or a    prefix of the
           input).    The rule is chosen as described    above in "How
           the Input is Matched", and yyyyyyyytttteeeexxxxtttt and yyyyyyyylllleeeennnngggg set    up
           appropriately.  It may either be    one which matched as
           much text as the    originally chosen rule but came    later
           in the _f_l_e_x input file, or one which matched less text.
           For example, the    following will both count the words in
           the input and call the routine special()    whenever
           "frob" is seen:

               int word_count = 0;
           %%

           frob           special(); REJECT;
           [^ \t\n]+   ++word_count;

           Without the RRRREEEEJJJJEEEECCCCTTTT,,,, any "frob"'s    in the input would not
           be counted as words, since the scanner normally
           executes    only one action    per token.  Multiple RRRREEEEJJJJEEEECCCCTTTT''''ssss
           are allowed, each one finding the next best choice to
           the currently active rule.  For example,    when the
           following scanner scans the token "abcd", it will write
           "abcdabcaba" to the output:

           %%
           a        |
           ab        |
           abc        |
           abcd        ECHO; REJECT;
           .|\n        /* eat up any unmatched character */

           (The first three    rules share the    fourth's action    since
           they use    the special '|'    action.)  RRRREEEEJJJJEEEECCCCTTTT is a
           particularly expensive feature in terms of scanner



     Page 12                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           performance; if it is used in _a_n_y of the    scanner's
           actions it will slow down _a_l_l of    the scanner's
           matching.  Furthermore, RRRREEEEJJJJEEEECCCCTTTT cannot be    used with the
           -_C_f or -_C_F options (see below).

           Note also that unlike the other special actions,    RRRREEEEJJJJEEEECCCCTTTT
           is a _b_r_a_n_c_h; code immediately following it in the
           action will _n_o_t be executed.

      -    yyyyyyyymmmmoooorrrreeee(((())))    tells the scanner that the next    time it
           matches a rule, the corresponding token should be
           _a_p_p_e_n_d_e_d    onto the current value of yyyyyyyytttteeeexxxxtttt rather    than
           replacing it.  For example, given the input "mega-
           kludge" the following will write    "mega-mega-kludge" to
           the output:

           %%
           mega-    ECHO; yymore();
           kludge   ECHO;

           First "mega-" is    matched    and echoed to the output.
           Then "kludge" is    matched, but the previous "mega-" is
           still hanging around at the beginning of    yyyyyyyytttteeeexxxxtttt so the
           EEEECCCCHHHHOOOO for    the "kludge" rule will actually    write "mega-
           kludge".

      Two notes regarding use of yyyyyyyymmmmoooorrrreeee(((()))).... First, yyyyyyyymmmmoooorrrreeee(((()))) depends
      on the value of _y_y_l_e_n_g correctly reflecting the size of the
      current token, so you    must not modify    _y_y_l_e_n_g if you are
      using    yyyyyyyymmmmoooorrrreeee(((()))).... Second, the presence of yyyyyyyymmmmoooorrrreeee(((()))) in the
      scanner's action entails a minor performance penalty in the
      scanner's matching speed.

      -    yyyyyyyylllleeeessssssss((((nnnn)))) returns all but the first _n characters    of the
           current token back to the input stream, where they will
           be rescanned when the scanner looks for the next    match.
           yyyyyyyytttteeeexxxxtttt and yyyyyyyylllleeeennnngggg are adjusted appropriately (e.g.,
           yyyyyyyylllleeeennnngggg will now be equal    to _n ).     For example, on the
           input "foobar" the following will write out
           "foobarbar":

           %%
           foobar    ECHO; yyless(3);
           [a-z]+    ECHO;

           An argument of 0    to yyyyyyyylllleeeessssssss will cause the entire
           current input string to be scanned again.  Unless
           you've changed how the scanner will subsequently
           process its input (using    BBBBEEEEGGGGIIIINNNN,,,, for example), this will
           result in an endless loop.

      Note that yyyyyyyylllleeeessssssss is a    macro and can only be used in the flex



     Page 13                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      input    file, not from other source files.

      -    uuuunnnnppppuuuutttt((((cccc))))    puts the character _c back onto the input
           stream.    It will    be the next character scanned.    The
           following action    will take the current token and    cause
           it to be    rescanned enclosed in parentheses.

           {
           int i;
           /* Copy yytext because unput() trashes yytext */
           char    *yycopy    = strdup( yytext );
           unput( ')' );
           for ( i = yyleng - 1; i >= 0; --i )
               unput( yycopy[i]    );
           unput( '(' );
           free( yycopy    );
           }

           Note that since each uuuunnnnppppuuuutttt(((()))) puts the given character
           back at the _b_e_g_i_n_n_i_n_g of    the input stream, pushing back
           strings must be done back-to-front.

      An important potential problem when using uuuunnnnppppuuuutttt(((()))) is that if
      you are using    %%%%ppppooooiiiinnnntttteeeerrrr (the default),    a call to uuuunnnnppppuuuutttt(((())))
      _d_e_s_t_r_o_y_s the contents    of _y_y_t_e_x_t, starting with its rightmost
      character and    devouring one character    to the left with each
      call.     If you    need the value of yytext preserved after a
      call to uuuunnnnppppuuuutttt(((()))) (as in the above example), you must either
      first    copy it    elsewhere, or build your scanner using %%%%aaaarrrrrrrraaaayyyy
      instead (see How The Input Is    Matched).

      Finally, note    that you cannot    put back EEEEOOOOFFFF to    attempt    to
      mark the input stream    with an    end-of-file.

      -    iiiinnnnppppuuuutttt(((()))) reads the next character    from the input stream.
           For example, the    following is one way to    eat up C
           comments:

           %%
           "/*"           {
                   register    int c;

                   for ( ; ; )
                   {
                   while ( (c =    input()) != '*'    &&
                       c !=    EOF )
                       ;    /* eat up text of comment */

                   if (    c == '*' )
                       {
                       while ( (c = input()) ==    '*' )
                       ;



     Page 14                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



                       if ( c == '/' )
                       break;    /*    found the end */
                       }

                   if (    c == EOF )
                       {
                       error( "EOF in comment" );
                       break;
                       }
                   }
                   }

           (Note that if the scanner is compiled using CCCC++++++++,,,,    then
           iiiinnnnppppuuuutttt(((()))) is instead referred to as yyyyyyyyiiiinnnnppppuuuutttt(((()))),,,, in order
           to avoid    a name clash with the CCCC++++++++ stream by the    name
           of _i_n_p_u_t.)

      -    YYYYYYYY____FFFFLLLLUUUUSSSSHHHH____BBBBUUUUFFFFFFFFEEEERRRR flushes the scanner's internal buffer
           so that the next    time the scanner attempts to match a
           token, it will first refill the buffer using YYYYYYYY____IIIINNNNPPPPUUUUTTTT
           (see The    Generated Scanner, below).  This action    is a
           special case of the more    general    yyyyyyyy____fffflllluuuusssshhhh____bbbbuuuuffffffffeeeerrrr(((())))
           function, described below in the    section    Multiple Input
           Buffers.

      -    yyyyyyyytttteeeerrrrmmmmiiiinnnnaaaatttteeee(((()))) can be used in lieu of a return statement
           in an action.  It terminates the    scanner    and returns a
           0 to the    scanner's caller, indicating "all done".  By
           default,    yyyyyyyytttteeeerrrrmmmmiiiinnnnaaaatttteeee(((()))) is also called when an end-of-
           file is encountered.  It    is a macro and may be
           redefined.

     TTTTHHHHEEEE GGGGEEEENNNNEEEERRRRAAAATTTTEEEEDDDD SSSSCCCCAAAANNNNNNNNEEEERRRR
      The output of    _f_l_e_x is    the file lllleeeexxxx....yyyyyyyy....cccc,,,, which contains the
      scanning routine yyyyyyyylllleeeexxxx(((()))),,,, a number of    tables used by it for
      matching tokens, and a number    of auxiliary routines and
      macros.  By default, yyyyyyyylllleeeexxxx(((()))) is declared as follows:

          int yylex()
          {
          ... various definitions and the actions in here ...
          }

      (If your environment supports    function prototypes, then it
      will be "int yylex( void )".)     This definition may be
      changed by defining the "YY_DECL" macro.  For    example, you
      could    use:

          #define YY_DECL float lexscan( a,    b ) float a, b;

      to give the scanning routine the name    _l_e_x_s_c_a_n, returning a
      float, and taking two    floats as arguments.  Note that    if you



     Page 15                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      give arguments to the    scanning routine using a K&R-
      style/non-prototyped function    declaration, you must
      terminate the    definition with    a semi-colon (;).

      Whenever yyyyyyyylllleeeexxxx(((()))) is called, it scans tokens from the global
      input    file _y_y_i_n (which defaults to stdin).  It continues
      until    it either reaches an end-of-file (at which point it
      returns the value 0) or one of its actions executes a    _r_e_t_u_r_n
      statement.

      If the scanner reaches an end-of-file, subsequent calls are
      undefined unless either _y_y_i_n is pointed at a new input file
      (in which case scanning continues from that file), or
      yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((()))) is called.  yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((()))) takes one    argument, a
      FFFFIIIILLLLEEEE **** pointer (which    can be nil, if you've set up YYYYYYYY____IIIINNNNPPPPUUUUTTTT
      to scan from a source    other than _y_y_i_n), and initializes _y_y_i_n
      for scanning from that file.    Essentially there is no
      difference between just assigning _y_y_i_n to a new input    file
      or using yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((()))) to do so; the latter is available for
      compatibility    with previous versions of _f_l_e_x,    and because it
      can be used to switch    input files in the middle of scanning.
      It can also be used to throw away the    current    input buffer,
      by calling it    with an    argument of _y_y_i_n; but better is    to use
      YYYYYYYY____FFFFLLLLUUUUSSSSHHHH____BBBBUUUUFFFFFFFFEEEERRRR (see above).    Note that yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((()))) does _n_o_t
      reset    the start condition to IIIINNNNIIIITTTTIIIIAAAALLLL (see Start Conditions,
      below).

      If yyyyyyyylllleeeexxxx(((()))) stops scanning due    to executing a _r_e_t_u_r_n
      statement in one of the actions, the scanner may then    be
      called again and it will resume scanning where it left off.

      By default (and for purposes of efficiency), the scanner
      uses block-reads rather than simple _g_e_t_c() calls to read
      characters from _y_y_i_n.    The nature of how it gets its input
      can be controlled by defining    the YYYYYYYY____IIIINNNNPPPPUUUUTTTT macro.
      YY_INPUT's calling sequence is
      "YY_INPUT(buf,result,max_size)".  Its    action is to place up
      to _m_a_x__s_i_z_e characters in the    character array    _b_u_f and    return
      in the integer variable _r_e_s_u_l_t either    the number of
      characters read or the constant YY_NULL (0 on    Unix systems)
      to indicate EOF.  The    default    YY_INPUT reads from the    global
      file-pointer "yyin".

      A sample definition of YY_INPUT (in the definitions section
      of the input file):

          %{
          #define YY_INPUT(buf,result,max_size) \
          { \
          int c    = getchar(); \
          result = (c == EOF) ?    YY_NULL    : (buf[0] = c, 1); \
          }



     Page 16                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          %}

      This definition will change the input    processing to occur
      one character    at a time.

      When the scanner receives an end-of-file indication from
      YY_INPUT, it then checks the yyyyyyyywwwwrrrraaaapppp(((())))    function.  If yyyyyyyywwwwrrrraaaapppp(((())))
      returns false    (zero),    then it    is assumed that    the function
      has gone ahead and set up _y_y_i_n to point to another input
      file,    and scanning continues.     If it returns true (non-
      zero), then the scanner terminates, returning    0 to its
      caller.  Note    that in    either case, the start condition
      remains unchanged; it    does _n_o_t revert    to IIIINNNNIIIITTTTIIIIAAAALLLL....

      If you do not    supply your own    version    of yyyyyyyywwwwrrrraaaapppp(((()))),,,, then you
      must either use %%%%ooooppppttttiiiioooonnnn nnnnooooyyyyyyyywwwwrrrraaaapppp (in which case the scanner
      behaves as though yyyyyyyywwwwrrrraaaapppp(((()))) returned 1), or you must link
      with ----llllffffllll to obtain the default version of the routine,
      which    always returns 1.

      Three    routines are available for scanning from in-memory
      buffers rather than files:  yyyyyyyy____ssssccccaaaannnn____ssssttttrrrriiiinnnngggg(((()))),,,,
      yyyyyyyy____ssssccccaaaannnn____bbbbyyyytttteeeessss(((()))),,,, and yyyyyyyy____ssssccccaaaannnn____bbbbuuuuffffffffeeeerrrr(((()))).... See the discussion of
      them below in    the section Multiple Input Buffers.

      The scanner writes its EEEECCCCHHHHOOOO output to    the _y_y_o_u_t global
      (default, stdout), which may be redefined by the user    simply
      by assigning it to some other    FFFFIIIILLLLEEEE pointer.

     SSSSTTTTAAAARRRRTTTT CCCCOOOONNNNDDDDIIIITTTTIIIIOOOONNNNSSSS
      _f_l_e_x provides    a mechanism for    conditionally activating
      rules.  Any rule whose pattern is prefixed with "<sc>" will
      only be active when the scanner is in    the start condition
      named    "sc".  For example,

          <STRING>[^"]*       { /*    eat up the string body ... */
              ...
              }

      will be active only when the scanner is in the "STRING"
      start    condition, and

          <INITIAL,STRING,QUOTE>\.          {    /* handle an escape ...    */
              ...
              }

      will be active only when the current start condition is
      either "INITIAL", "STRING", or "QUOTE".

      Start    conditions are declared    in the definitions (first)
      section of the input using unindented    lines beginning    with
      either %%%%ssss or %%%%xxxx followed by a    list of    names.    The former



     Page 17                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      declares _i_n_c_l_u_s_i_v_e start conditions, the latter _e_x_c_l_u_s_i_v_e
      start    conditions.  A start condition is activated using the
      BBBBEEEEGGGGIIIINNNN    action.     Until the next    BBBBEEEEGGGGIIIINNNN action is    executed,
      rules    with the given start condition will be active and
      rules    with other start conditions will be inactive.  If the
      start    condition is _i_n_c_l_u_s_i_v_e,    then rules with    no start
      conditions at    all will also be active.  If it    is _e_x_c_l_u_s_i_v_e,
      then _o_n_l_y rules qualified with the start condition will be
      active.  A set of rules contingent on    the same exclusive
      start    condition describe a scanner which is independent of
      any of the other rules in the    _f_l_e_x input.  Because of    this,
      exclusive start conditions make it easy to specify "mini-
      scanners" which scan portions    of the input that are
      syntactically    different from the rest    (e.g., comments).

      If the distinction between inclusive and exclusive start
      conditions is    still a    little vague, here's a simple example
      illustrating the connection between the two.    The set    of
      rules:

          %s example
          %%

          <example>foo   do_something();

          bar         something_else();

      is equivalent    to

          %x example
          %%

          <example>foo   do_something();

          <INITIAL,example>bar    something_else();

      Without the <<<<IIIINNNNIIIITTTTIIIIAAAALLLL,,,,eeeexxxxaaaammmmpppplllleeee>>>>    qualifier, the _b_a_r pattern in
      the second example wouldn't be active    (i.e., couldn't    match)
      when in start    condition eeeexxxxaaaammmmpppplllleeee.... If we just used <<<<eeeexxxxaaaammmmpppplllleeee>>>>
      to qualify _b_a_r, though, then it would    only be    active in
      eeeexxxxaaaammmmpppplllleeee and not in IIIINNNNIIIITTTTIIIIAAAALLLL,,,, while in the first example it's
      active in both, because in the first example the eeeexxxxaaaammmmpppplllleeee
      startion condition is    an _i_n_c_l_u_s_i_v_e ((((%%%%ssss)))) start    condition.

      Also note that the special start-condition specifier <<<<****>>>>
      matches every    start condition.  Thus,    the above example
      could    also have been written;

          %x example
          %%

          <example>foo   do_something();



     Page 18                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          <*>bar    something_else();


      The default rule (to EEEECCCCHHHHOOOO any    unmatched character) remains
      active in start conditions.  It is equivalent    to:

          <*>.|\n      ECHO;


      BBBBEEEEGGGGIIIINNNN((((0000)))) returns to the original state where only the    rules
      with no start    conditions are active.    This state can also be
      referred to as the start-condition "INITIAL",    so
      BBBBEEEEGGGGIIIINNNN((((IIIINNNNIIIITTTTIIIIAAAALLLL)))) is equivalent to BBBBEEEEGGGGIIIINNNN((((0000)))).... (The parentheses
      around the start condition name are not required but are
      considered good style.)

      BBBBEEEEGGGGIIIINNNN    actions    can also be given as indented code at the
      beginning of the rules section.  For example,    the following
      will cause the scanner to enter the "SPECIAL"    start
      condition whenever yyyyyyyylllleeeexxxx(((()))) is    called and the global variable
      _e_n_t_e_r__s_p_e_c_i_a_l    is true:

              int enter_special;

          %x SPECIAL
          %%
              if ( enter_special )
              BEGIN(SPECIAL);

          <SPECIAL>blahblahblah
          ...more rules follow...


      To illustrate    the uses of start conditions, here is a
      scanner which    provides two different interpretations of a
      string like "123.456".  By default it    will treat it as three
      tokens, the integer "123", a dot ('.'), and the integer
      "456".  But if the string is preceded    earlier    in the line by
      the string "expect-floats" it    will treat it as a single
      token, the floating-point number 123.456:

          %{
          #include <math.h>
          %}
          %s expect

          %%
          expect-floats       BEGIN(expect);

          <expect>[0-9]+"."[0-9]+       {
              printf( "found a float, = %f\n",
                  atof(    yytext ) );



     Page 19                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



              }
          <expect>\n       {
              /* that's the    end of the line, so
               * we    need another "expect-number"
               * before we'll recognize any    more
               * numbers
               */
              BEGIN(INITIAL);
              }

          [0-9]+      {
              printf( "found an integer, = %d\n",
                  atoi(    yytext ) );
              }

          "."      printf( "found a dot\n" );

      Here is a scanner which recognizes (and discards) C comments
      while    maintaining a count of the current input line.

          %x comment
          %%
              int line_num = 1;

          "/*"       BEGIN(comment);

          <comment>[^*\n]*          /* eat anything that's not a '*' */
          <comment>"*"+[^*/\n]*   /* eat up    '*'s not followed by '/'s */
          <comment>\n          ++line_num;
          <comment>"*"+"/"          BEGIN(INITIAL);

      This scanner goes to a bit of    trouble    to match as much text
      as possible with each    rule.  In general, when    attempting to
      write    a high-speed scanner try to match as much possible in
      each rule, as    it's a big win.

      Note that start-conditions names are really integer values
      and can be stored as such.  Thus, the    above could be
      extended in the following fashion:

          %x comment foo
          %%
              int line_num = 1;
              int comment_caller;

          "/*"       {
               comment_caller = INITIAL;
               BEGIN(comment);
               }

          ...




     Page 20                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          <foo>"/*"       {
               comment_caller = foo;
               BEGIN(comment);
               }

          <comment>[^*\n]*          /* eat anything that's not a '*' */
          <comment>"*"+[^*/\n]*   /* eat up    '*'s not followed by '/'s */
          <comment>\n          ++line_num;
          <comment>"*"+"/"          BEGIN(comment_caller);

      Furthermore, you can access the current start    condition
      using    the integer-valued YYYYYYYY____SSSSTTTTAAAARRRRTTTT macro.  For    example, the
      above    assignments to _c_o_m_m_e_n_t__c_a_l_l_e_r could instead be written

          comment_caller = YY_START;

      Flex provides    YYYYYYYYSSSSTTTTAAAATTTTEEEE    as an alias for    YYYYYYYY____SSSSTTTTAAAARRRRTTTT (since    that
      is what's used by AT&T _l_e_x).

      Note that start conditions do    not have their own name-space;
      %s's and %x's    declare    names in the same fashion as
      #define's.

      Finally, here's an example of    how to match C-style quoted
      strings using    exclusive start    conditions, including expanded
      escape sequences (but    not including checking for a string
      that's too long):

          %x str

          %%
              char string_buf[MAX_STR_CONST];
              char *string_buf_ptr;


          \"      string_buf_ptr = string_buf; BEGIN(str);

          <str>\"         { /* saw closing quote - all done */
              BEGIN(INITIAL);
              *string_buf_ptr =    '\0';
              /* return    string constant    token type and
               * value to parser
               */
              }

          <str>\n         {
              /* error - unterminated string constant */
              /* generate error    message    */
              }

          <str>\\[0-7]{1,3}    {
              /* octal escape sequence */



     Page 21                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



              int result;

              (void) sscanf( yytext + 1, "%o", &result );

              if ( result > 0xff )
                  /* error,    constant is out-of-bounds */

              *string_buf_ptr++    = result;
              }

          <str>\\[0-9]+ {
              /* generate error    - bad escape sequence; something
               * like '\48' or '\0777777'
               */
              }

          <str>\\n    *string_buf_ptr++ = '\n';
          <str>\\t    *string_buf_ptr++ = '\t';
          <str>\\r    *string_buf_ptr++ = '\r';
          <str>\\b    *string_buf_ptr++ = '\b';
          <str>\\f    *string_buf_ptr++ = '\f';

          <str>\\(.|\n)  *string_buf_ptr++ = yytext[1];

          <str>[^\\\n\"]+         {
              char *yptr = yytext;

              while ( *yptr )
                  *string_buf_ptr++    = *yptr++;
              }


      Often, such as in some of the    examples above,    you wind up
      writing a whole bunch    of rules all preceded by the same
      start    condition(s).  Flex makes this a little    easier and
      cleaner by introducing a notion of start condition _s_c_o_p_e. A
      start    condition scope    is begun with:

          <SCs>{

      where    _S_C_s is a list of one or    more start conditions.    Inside
      the start condition scope, every rule    automatically has the
      prefix <_S_C_s> applied to it, until a '}' which    matches    the
      initial '{'. So, for example,

          <ESC>{
          "\\n"      return '\n';
          "\\r"      return '\r';
          "\\f"      return '\f';
          "\\0"      return '\0';
          }




     Page 22                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      is equivalent    to:

          <ESC>"\\n"  return '\n';
          <ESC>"\\r"  return '\r';
          <ESC>"\\f"  return '\f';
          <ESC>"\\0"  return '\0';

      Start    condition scopes may be    nested.

      Three    routines are available for manipulating    stacks of
      start    conditions:

      vvvvooooiiiidddd yyyyyyyy____ppppuuuusssshhhh____ssssttttaaaatttteeee((((iiiinnnntttt nnnneeeewwww____ssssttttaaaatttteeee))))
           pushes the current start    condition onto the top of the
           start condition stack and switches to _n_e_w__s_t_a_t_e as
           though you had used BBBBEEEEGGGGIIIINNNN nnnneeeewwww____ssssttttaaaatttteeee (recall that    start
           condition names are also    integers).

      vvvvooooiiiidddd yyyyyyyy____ppppoooopppp____ssssttttaaaatttteeee(((())))
           pops the    top of the stack and switches to it via    BBBBEEEEGGGGIIIINNNN....

      iiiinnnntttt yyyyyyyy____ttttoooopppp____ssssttttaaaatttteeee(((())))
           returns the top of the stack without altering the
           stack's contents.

      The start condition stack grows dynamically and so has no
      built-in size    limitation.  If    memory is exhausted, program
      execution aborts.

      To use start condition stacks, your scanner must include a
      %%%%ooooppppttttiiiioooonnnn ssssttttaaaacccckkkk    directive (see Options below).

     MMMMUUUULLLLTTTTIIIIPPPPLLLLEEEE IIIINNNNPPPPUUUUTTTT BBBBUUUUFFFFFFFFEEEERRRRSSSS
      Some scanners    (such as those which support "include" files)
      require reading from several input streams.  As _f_l_e_x
      scanners do a    large amount of    buffering, one cannot control
      where    the next input will be read from by simply writing a
      YYYYYYYY____IIIINNNNPPPPUUUUTTTT which is sensitive to the scanning context.
      YYYYYYYY____IIIINNNNPPPPUUUUTTTT is only called when the scanner reaches the end of
      its buffer, which may    be a long time after scanning a
      statement such as an "include" which requires    switching the
      input    source.

      To negotiate these sorts of problems,    _f_l_e_x provides a
      mechanism for    creating and switching between multiple    input
      buffers.  An input buffer is created by using:

          YY_BUFFER_STATE yy_create_buffer(    FILE *file, int    size )

      which    takes a    _F_I_L_E pointer and a size    and creates a buffer
      associated with the given file and large enough to hold _s_i_z_e
      characters (when in doubt, use YYYYYYYY____BBBBUUUUFFFF____SSSSIIIIZZZZEEEE for the size).



     Page 23                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      It returns a YYYYYYYY____BBBBUUUUFFFFFFFFEEEERRRR____SSSSTTTTAAAATTTTEEEE handle, which may then be
      passed to other routines (see    below).     The YYYYYYYY____BBBBUUUUFFFFFFFFEEEERRRR____SSSSTTTTAAAATTTTEEEE
      type is a pointer to an opaque ssssttttrrrruuuucccctttt    yyyyyyyy____bbbbuuuuffffffffeeeerrrr____ssssttttaaaatttteeee
      structure, so    you may    safely initialize YY_BUFFER_STATE
      variables to ((((((((YYYYYYYY____BBBBUUUUFFFFFFFFEEEERRRR____SSSSTTTTAAAATTTTEEEE)))) 0000)))) if    you wish, and also
      refer    to the opaque structure    in order to correctly declare
      input    buffers    in source files    other than that    of your
      scanner.  Note that the _F_I_L_E pointer in the call to
      yyyyyyyy____ccccrrrreeeeaaaatttteeee____bbbbuuuuffffffffeeeerrrr is only used    as the value of    _y_y_i_n seen by
      YYYYYYYY____IIIINNNNPPPPUUUUTTTT;;;; if you redefine YYYYYYYY____IIIINNNNPPPPUUUUTTTT so    it no longer uses
      _y_y_i_n,    then you can safely pass a nil _F_I_L_E pointer to
      yyyyyyyy____ccccrrrreeeeaaaatttteeee____bbbbuuuuffffffffeeeerrrr.... You    select a particular buffer to scan
      from using:

          void yy_switch_to_buffer(    YY_BUFFER_STATE    new_buffer )

      switches the scanner's input buffer so subsequent tokens
      will come from _n_e_w__b_u_f_f_e_r. Note that yyyyyyyy____sssswwwwiiiittttcccchhhh____ttttoooo____bbbbuuuuffffffffeeeerrrr(((())))
      may be used by yywrap() to set things    up for continued
      scanning, instead of opening a new file and pointing _y_y_i_n at
      it.  Note also that switching    input sources via either
      yyyyyyyy____sssswwwwiiiittttcccchhhh____ttttoooo____bbbbuuuuffffffffeeeerrrr(((())))    or yyyyyyyywwwwrrrraaaapppp(((()))) does _n_o_t change the    start
      condition.

          void yy_delete_buffer( YY_BUFFER_STATE buffer )

      is used to reclaim the storage associated with a buffer.  (
      bbbbuuuuffffffffeeeerrrr can be    nil, in    which case the routine does nothing.)
      You can also clear the current contents of a buffer using:

          void yy_flush_buffer( YY_BUFFER_STATE buffer )

      This function    discards the buffer's contents,    so the next
      time the scanner attempts to match a token from the buffer,
      it will first    fill the buffer    anew using YYYYYYYY____IIIINNNNPPPPUUUUTTTT....

      yyyyyyyy____nnnneeeewwww____bbbbuuuuffffffffeeeerrrr(((()))) is an    alias for yyyyyyyy____ccccrrrreeeeaaaatttteeee____bbbbuuuuffffffffeeeerrrr(((()))),,,, provided
      for compatibility with the C++ use of    _n_e_w and    _d_e_l_e_t_e for
      creating and destroying dynamic objects.

      Finally, the YYYYYYYY____CCCCUUUURRRRRRRREEEENNNNTTTT____BBBBUUUUFFFFFFFFEEEERRRR macro returns a
      YYYYYYYY____BBBBUUUUFFFFFFFFEEEERRRR____SSSSTTTTAAAATTTTEEEE handle to the    current    buffer.

      Here is an example of    using these features for writing a
      scanner which    expands    include    files (the <<<<<<<<EEEEOOOOFFFF>>>>>>>> feature is
      discussed below):

          /* the "incl" state is used for picking up the name
           * of an include file
           */
          %x incl




     Page 24                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          %{
          #define MAX_INCLUDE_DEPTH    10
          YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH];
          int include_stack_ptr = 0;
          %}

          %%
          include          BEGIN(incl);

          [a-z]+          ECHO;
          [^a-z\n]*\n?      ECHO;

          <incl>[ \t]*    /* eat the whitespace */
          <incl>[^ \t\n]+    { /* got the include file name */
              if ( include_stack_ptr >=    MAX_INCLUDE_DEPTH )
              {
              fprintf( stderr, "Includes nested too    deeply"    );
              exit(    1 );
              }

              include_stack[include_stack_ptr++] =
              YY_CURRENT_BUFFER;

              yyin = fopen( yytext, "r"    );

              if ( ! yyin )
              error( ... );

              yy_switch_to_buffer(
              yy_create_buffer( yyin, YY_BUF_SIZE )    );

              BEGIN(INITIAL);
              }

          <<EOF>> {
              if ( --include_stack_ptr < 0 )
              {
              yyterminate();
              }

              else
              {
              yy_delete_buffer( YY_CURRENT_BUFFER );
              yy_switch_to_buffer(
                   include_stack[include_stack_ptr]    );
              }
              }

      Three    routines are available for setting up input buffers
      for scanning in-memory strings instead of files.  All    of
      them create a    new input buffer for scanning the string, and
      return a corresponding YYYYYYYY____BBBBUUUUFFFFFFFFEEEERRRR____SSSSTTTTAAAATTTTEEEE handle    (which you



     Page 25                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      should delete    with yyyyyyyy____ddddeeeelllleeeetttteeee____bbbbuuuuffffffffeeeerrrr(((())))    when done with it).
      They also switch to the new buffer using
      yyyyyyyy____sssswwwwiiiittttcccchhhh____ttttoooo____bbbbuuuuffffffffeeeerrrr(((()))),,,, so the    next call to yyyyyyyylllleeeexxxx(((()))) will
      start    scanning the string.

      yyyyyyyy____ssssccccaaaannnn____ssssttttrrrriiiinnnngggg((((ccccoooonnnnsssstttt cccchhhhaaaarrrr ****ssssttttrrrr))))
           scans a NUL-terminated string.

      yyyyyyyy____ssssccccaaaannnn____bbbbyyyytttteeeessss((((ccccoooonnnnsssstttt cccchhhhaaaarrrr ****bbbbyyyytttteeeessss,,,, iiiinnnntttt lllleeeennnn))))
           scans _l_e_n bytes (including possibly NUL's) starting at
           location    _b_y_t_e_s.

      Note that both of these functions create and scan a _c_o_p_y of
      the string or    bytes.    (This may be desirable,    since yyyyyyyylllleeeexxxx(((())))
      modifies the contents    of the buffer it is scanning.)    You
      can avoid the    copy by    using:

      yyyyyyyy____ssssccccaaaannnn____bbbbuuuuffffffffeeeerrrr((((cccchhhhaaaarrrr ****bbbbaaaasssseeee,,,, yyyyyyyy____ssssiiiizzzzeeee____tttt ssssiiiizzzzeeee))))
           which scans in place the    buffer starting    at _b_a_s_e,
           consisting of _s_i_z_e bytes, the last two bytes of which
           _m_u_s_t be YYYYYYYY____EEEENNNNDDDD____OOOOFFFF____BBBBUUUUFFFFFFFFEEEERRRR____CCCCHHHHAAAARRRR (ASCII NUL).  These last
           two bytes are not scanned; thus,    scanning consists of
           bbbbaaaasssseeee[[[[0000]]]] through bbbbaaaasssseeee[[[[ssssiiiizzzzeeee----2222]]]],,,, inclusive.

           If you fail to set up _b_a_s_e in this manner (i.e.,    forget
           the final two YYYYYYYY____EEEENNNNDDDD____OOOOFFFF____BBBBUUUUFFFFFFFFEEEERRRR____CCCCHHHHAAAARRRR bytes), then
           yyyyyyyy____ssssccccaaaannnn____bbbbuuuuffffffffeeeerrrr(((())))    returns    a nil pointer instead of
           creating    a new input buffer.

           The type    yyyyyyyy____ssssiiiizzzzeeee____tttt is an    integral type to which you can
           cast an integer expression reflecting the size of the
           buffer.

     EEEENNNNDDDD----OOOOFFFF----FFFFIIIILLLLEEEE RRRRUUUULLLLEEEESSSS
      The special rule "<<EOF>>" indicates actions which are to be
      taken    when an    end-of-file is encountered and yywrap()
      returns non-zero (i.e., indicates no further files to
      process).  The action    must finish by doing one of four
      things:

      -    assigning _y_y_i_n to a new input file (in previous
           versions    of flex, after doing the assignment you    had to
           call the    special    action YYYYYYYY____NNNNEEEEWWWW____FFFFIIIILLLLEEEE;;;; this is no longer
           necessary);

      -    executing a _r_e_t_u_r_n statement;

      -    executing the special yyyyyyyytttteeeerrrrmmmmiiiinnnnaaaatttteeee(((()))) action;

      -    or, switching to    a new buffer using
           yyyyyyyy____sssswwwwiiiittttcccchhhh____ttttoooo____bbbbuuuuffffffffeeeerrrr(((()))) as    shown in the example above.




     Page 26                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      <<EOF>> rules    may not    be used    with other patterns; they may
      only be qualified with a list    of start conditions.  If an
      unqualified <<EOF>> rule is given, it    applies    to _a_l_l start
      conditions which do not already have <<EOF>> actions.     To
      specify an <<EOF>> rule for only the initial start
      condition, use

          <INITIAL><<EOF>>


      These    rules are useful for catching things like unclosed
      comments.  An    example:

          %x quote
          %%

          ...other rules for dealing with quotes...

          <quote><<EOF>>   {
               error( "unterminated quote" );
               yyterminate();
               }
          <<EOF>>  {
               if ( *++filelist    )
               yyin    = fopen( *filelist, "r"    );
               else
              yyterminate();
               }


     MMMMIIIISSSSCCCCEEEELLLLLLLLAAAANNNNEEEEOOOOUUUUSSSS MMMMAAAACCCCRRRROOOOSSSS
      The macro YYYYYYYY____UUUUSSSSEEEERRRR____AAAACCCCTTTTIIIIOOOONNNN can be defined to provide an    action
      which    is always executed prior to the    matched    rule's action.
      For example, it could    be #define'd to    call a routine to
      convert yytext to lower-case.     When YYYYYYYY____UUUUSSSSEEEERRRR____AAAACCCCTTTTIIIIOOOONNNN is
      invoked, the variable    _y_y__a_c_t gives the number    of the matched
      rule (rules are numbered starting with 1).  Suppose you want
      to profile how often each of your rules is matched.  The
      following would do the trick:

          #define YY_USER_ACTION ++ctr[yy_act]

      where    _c_t_r is an array    to hold    the counts for the different
      rules.  Note that the    macro YYYYYYYY____NNNNUUUUMMMM____RRRRUUUULLLLEEEESSSS gives the total
      number of rules (including the default rule, even if you use
      ----ssss)))),,,, so a correct declaration    for _c_t_r    is:

          int ctr[YY_NUM_RULES];


      The macro YYYYYYYY____UUUUSSSSEEEERRRR____IIIINNNNIIIITTTT may be    defined    to provide an action
      which    is always executed before the first scan (and before



     Page 27                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      the scanner's    internal initializations are done).  For
      example, it could be used to call a routine to read in a
      data table or    open a logging file.

      The macro yyyyyyyy____sssseeeetttt____iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee((((iiiissss____iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee)))) can be used to
      control whether the current buffer is    considered
      _i_n_t_e_r_a_c_t_i_v_e. An interactive buffer is    processed more slowly,
      but must be used when    the scanner's input source is indeed
      interactive to avoid problems    due to waiting to fill buffers
      (see the discussion of the ----IIII    flag below).  A    non-zero value
      in the macro invocation marks    the buffer as interactive, a
      zero value as    non-interactive.  Note that use    of this    macro
      overrides %%%%ooooppppttttiiiioooonnnn aaaallllwwwwaaaayyyyssss----iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee or %%%%ooooppppttttiiiioooonnnn nnnneeeevvvveeeerrrr----
      iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee (see Options below).  yyyyyyyy____sssseeeetttt____iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee(((()))) must
      be invoked prior to beginning    to scan    the buffer that    is (or
      is not) to be    considered interactive.

      The macro yyyyyyyy____sssseeeetttt____bbbboooollll((((aaaatttt____bbbboooollll)))) can be used to control whether
      the current buffer's scanning    context    for the    next token
      match    is done    as though at the beginning of a    line.  A non-
      zero macro argument makes rules anchored with

      The macro YYYYYYYY____AAAATTTT____BBBBOOOOLLLL(((())))    returns    true if    the next token scanned
      from the current buffer will have '^'    rules active, false
      otherwise.

      In the generated scanner, the    actions    are all    gathered in
      one large switch statement and separated using YYYYYYYY____BBBBRRRREEEEAAAAKKKK,,,,
      which    may be redefined.  By default, it is simply a "break",
      to separate each rule's action from the following rule's.
      Redefining YYYYYYYY____BBBBRRRREEEEAAAAKKKK allows, for example, C++ users to
      #define YY_BREAK to do nothing (while    being very careful
      that every rule ends with a "break" or a "return"!) to avoid
      suffering from unreachable statement warnings    where because
      a rule's action ends with "return", the YYYYYYYY____BBBBRRRREEEEAAAAKKKK is
      inaccessible.

     VVVVAAAALLLLUUUUEEEESSSS AAAAVVVVAAAAIIIILLLLAAAABBBBLLLLEEEE TTTTOOOO TTTTHHHHEEEE UUUUSSSSEEEERRRR
      This section summarizes the various values available to the
      user in the rule actions.

      -    cccchhhhaaaarrrr ****yyyyyyyytttteeeexxxxtttt holds the text of the current token.  It
           may be modified but not lengthened (you cannot append
           characters to the end).

           If the special directive    %%%%aaaarrrrrrrraaaayyyy appears in the first
           section of the scanner description, then    yyyyyyyytttteeeexxxxtttt is
           instead declared    cccchhhhaaaarrrr yyyyyyyytttteeeexxxxtttt[[[[YYYYYYYYLLLLMMMMAAAAXXXX]]]],,,, where YYYYYYYYLLLLMMMMAAAAXXXX is a
           macro definition    that you can redefine in the first
           section if you don't like the default value (generally
           8KB).  Using %%%%aaaarrrrrrrraaaayyyy results in somewhat slower
           scanners, but the value of yyyyyyyytttteeeexxxxtttt becomes immune    to



     Page 28                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           calls to    _i_n_p_u_t()    and _u_n_p_u_t(), which potentially destroy
           its value when yyyyyyyytttteeeexxxxtttt is    a character pointer.  The
           opposite    of %%%%aaaarrrrrrrraaaayyyy is %%%%ppppooooiiiinnnntttteeeerrrr,,,, which is    the default.

           You cannot use %%%%aaaarrrrrrrraaaayyyy when generating C++ scanner
           classes (the ----++++ flag).

      -    iiiinnnntttt yyyyyyyylllleeeennnngggg holds    the length of the current token.

      -    FFFFIIIILLLLEEEE ****yyyyyyyyiiiinnnn is the file which by default _f_l_e_x reads
           from.  It may be    redefined but doing so only makes
           sense before scanning begins or after an    EOF has    been
           encountered.  Changing it in the    midst of scanning will
           have unexpected results since _f_l_e_x buffers its input;
           use yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((()))) instead.     Once scanning terminates
           because an end-of-file has been seen, you can assign
           _y_y_i_n at the new input file and then call    the scanner
           again to    continue scanning.

      -    vvvvooooiiiidddd yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((( FFFFIIIILLLLEEEE ****nnnneeeewwww____ffffiiiilllleeee ))))    may be called to point
           _y_y_i_n at the new input file.  The    switch-over to the new
           file is immediate (any previously buffered-up input is
           lost).  Note that calling yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((()))) with _y_y_i_n as an
           argument    thus throws away the current input buffer and
           continues scanning the same input file.

      -    FFFFIIIILLLLEEEE ****yyyyyyyyoooouuuutttt is the file to which    EEEECCCCHHHHOOOO actions are done.
           It can be reassigned by the user.

      -    YYYYYYYY____CCCCUUUURRRRRRRREEEENNNNTTTT____BBBBUUUUFFFFFFFFEEEERRRR returns a YYYYYYYY____BBBBUUUUFFFFFFFFEEEERRRR____SSSSTTTTAAAATTTTEEEE handle to
           the current buffer.

      -    YYYYYYYY____SSSSTTTTAAAARRRRTTTT    returns    an integer value corresponding to the
           current start condition.     You can subsequently use this
           value with BBBBEEEEGGGGIIIINNNN    to return to that start    condition.

     IIIINNNNTTTTEEEERRRRFFFFAAAACCCCIIIINNNNGGGG WWWWIIIITTTTHHHH YYYYAAAACCCCCCCC
      One of the main uses of _f_l_e_x is as a companion to the    _y_a_c_c
      parser-generator.  _y_a_c_c parsers expect to call a routine
      named    yyyyyyyylllleeeexxxx(((())))    to find    the next input token.  The routine is
      supposed to return the type of the next token    as well    as
      putting any associated value in the global yyyyyyyyllllvvvvaaaallll.... To    use
      _f_l_e_x with _y_a_c_c, one specifies    the ----dddd option to _y_a_c_c to
      instruct it to generate the file yyyy....ttttaaaabbbb....hhhh containing
      definitions of all the %%%%ttttooookkkkeeeennnnssss appearing in the _y_a_c_c input.
      This file is then included in    the _f_l_e_x scanner.  For
      example, if one of the tokens    is "TOK_NUMBER", part of the
      scanner might    look like:

          %{
          #include "y.tab.h"
          %}



     Page 29                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          %%

          [0-9]+        yylval = atoi( yytext ); return TOK_NUMBER;


     OOOOPPPPTTTTIIIIOOOONNNNSSSS
      _f_l_e_x has the following options:

      ----bbbb   Generate    backing-up information to _l_e_x._b_a_c_k_u_p. This is
           a list of scanner states    which require backing up and
           the input characters on which they do so.  By adding
           rules one can remove backing-up states.    If _a_l_l
           backing-up states are eliminated    and ----CCCCffff    or ----CCCCFFFF is
           used, the generated scanner will    run faster (see    the ----pppp
           flag).  Only users who wish to squeeze every last cycle
           out of their scanners need worry    about this option.
           (See the    section    on Performance Considerations below.)

      ----cccc   is a do-nothing,    deprecated option included for POSIX
           compliance.

      ----dddd   makes the generated scanner run in _d_e_b_u_g    mode.
           Whenever    a pattern is recognized    and the    global
           yyyyyyyy____fffflllleeeexxxx____ddddeeeebbbbuuuugggg is    non-zero (which    is the default), the
           scanner will write to _s_t_d_e_r_r a line of the form:

           --accepting rule at line 53 ("the matched text")

           The line    number refers to the location of the rule in
           the file    defining the scanner (i.e., the    file that was
           fed to flex).  Messages are also    generated when the
           scanner backs up, accepts the default rule, reaches the
           end of its input    buffer (or encounters a    NUL; at    this
           point, the two look the same as far as the scanner's
           concerned), or reaches an end-of-file.

      ----ffff   specifies _f_a_s_t _s_c_a_n_n_e_r. No table    compression is done
           and stdio is bypassed.  The result is large but fast.
           This option is equivalent to ----CCCCffffrrrr (see below).

      ----hhhh   generates a "help" summary of _f_l_e_x'_s options to _s_t_d_o_u_t
           and then    exits.    ----???? and --------hhhheeeellllpppp are synonyms for ----hhhh....

      ----iiii   instructs _f_l_e_x to generate a _c_a_s_e-_i_n_s_e_n_s_i_t_i_v_e scanner.
           The case    of letters given in the    _f_l_e_x input patterns
           will be ignored,    and tokens in the input    will be
           matched regardless of case.  The    matched    text given in
           _y_y_t_e_x_t will have    the preserved case (i.e., it will not
           be folded).

      ----llll   turns on    maximum    compatibility with the original    AT&T
           _l_e_x implementation.  Note that this does    not mean _f_u_l_l



     Page 30                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           compatibility.  Use of this option costs    a considerable
           amount of performance, and it cannot be used with the
           ----++++,,,, ----ffff,,,, ----FFFF,,,, ----CCCCffff,,,,    or ----CCCCFFFF options.     For details on    the
           compatibilities it provides, see    the section
           "Incompatibilities With Lex And POSIX" below.  This
           option also results in the name YYYYYYYY____FFFFLLLLEEEEXXXX____LLLLEEEEXXXX____CCCCOOOOMMMMPPPPAAAATTTT
           being #define'd in the generated    scanner.

      ----nnnn   is another do-nothing, deprecated option    included only
           for POSIX compliance.

      ----pppp   generates a performance report to stderr.  The report
           consists    of comments regarding features of the _f_l_e_x
           input file which    will cause a serious loss of
           performance in the resulting scanner.  If you give the
           flag twice, you will also get comments regarding
           features    that lead to minor performance losses.

           Note that the use of RRRREEEEJJJJEEEECCCCTTTT,,,, %%%%ooooppppttttiiiioooonnnn yyyyyyyylllliiiinnnneeeennnnoooo,,,, and
           variable    trailing context (see the Deficiencies / Bugs
           section below) entails a    substantial performance
           penalty;    use of _y_y_m_o_r_e(), the ^^^^ operator, and the ----IIII
           flag entail minor performance penalties.

      ----ssss   causes the _d_e_f_a_u_l_t _r_u_l_e (that unmatched scanner input
           is echoed to _s_t_d_o_u_t) to be suppressed.  If the scanner
           encounters input    that does not match any    of its rules,
           it aborts with an error.     This option is    useful for
           finding holes in    a scanner's rule set.

      ----tttt   instructs _f_l_e_x to write the scanner it generates    to
           standard    output instead of lllleeeexxxx....yyyyyyyy....cccc....

      ----vvvv   specifies that _f_l_e_x should write    to _s_t_d_e_r_r a summary of
           statistics regarding the    scanner    it generates.  Most of
           the statistics are meaningless to the casual _f_l_e_x user,
           but the first line identifies the version of _f_l_e_x (same
           as reported by ----VVVV)))),,,, and the next    line the flags used
           when generating the scanner, including those that are
           on by default.

      ----wwww   suppresses warning messages.

      ----BBBB   instructs _f_l_e_x to generate a _b_a_t_c_h scanner, the
           opposite    of _i_n_t_e_r_a_c_t_i_v_e scanners    generated by ----IIII    (see
           below).    In general, you    use ----BBBB when you    are _c_e_r_t_a_i_n
           that your scanner will never be used interactively, and
           you want    to squeeze a _l_i_t_t_l_e more performance out of
           it.  If your goal is instead to squeeze out a _l_o_t more
           performance, you    should    be using the ----CCCCffff or ----CCCCFFFF
           options (discussed below), which    turn on    ----BBBB
           automatically anyway.



     Page 31                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      ----FFFF   specifies that the _f_a_s_t scanner table representation
           should be used (and stdio bypassed).  This
           representation is about as fast as the full table
           representation ((((----ffff)))),,,, and    for some sets of patterns will
           be considerably smaller (and for    others,    larger).  In
           general,    if the pattern set contains both "keywords"
           and a catch-all,    "identifier" rule, such    as in the set:

           "case"    return TOK_CASE;
           "switch"  return TOK_SWITCH;
           ...
           "default" return TOK_DEFAULT;
           [a-z]+    return TOK_ID;

           then you're better off using the    full table
           representation.    If only    the "identifier" rule is
           present and you then use    a hash table or    some such to
           detect the keywords, you're better off using ----FFFF....

           This option is equivalent to ----CCCCFFFFrrrr (see below).  It
           cannot be used with ----++++....

      ----IIII   instructs _f_l_e_x to generate an _i_n_t_e_r_a_c_t_i_v_e scanner.  An
           interactive scanner is one that only looks ahead    to
           decide what token has been matched if it    absolutely
           must.  It turns out that    always looking one extra
           character ahead,    even if    the scanner has    already    seen
           enough text to disambiguate the current token, is a bit
           faster than only    looking    ahead when necessary.  But
           scanners    that always look ahead give dreadful
           interactive performance;    for example, when a user types
           a newline, it is    not recognized as a newline token
           until they enter    _a_n_o_t_h_e_r    token, which often means
           typing in another whole line.

           _F_l_e_x scanners default to    _i_n_t_e_r_a_c_t_i_v_e unless you use the
           ----CCCCffff or ----CCCCFFFF table-compression options (see below).
           That's because if you're    looking    for high-performance
           you should be using one of these    options, so if you
           didn't, _f_l_e_x assumes you'd rather trade off a bit of
           run-time    performance for    intuitive interactive
           behavior.  Note also that you _c_a_n_n_o_t use    ----IIII in
           conjunction with    ----CCCCffff or ----CCCCFFFF.... Thus, this option is not
           really needed; it is on by default for all those    cases
           in which    it is allowed.

           You can force a scanner to _n_o_t be interactive by    using
           ----BBBB (see above).

      ----LLLL   instructs _f_l_e_x not to generate ####lllliiiinnnneeee directives.
           Without this option, _f_l_e_x peppers the generated scanner
           with #line directives so    error messages in the actions



     Page 32                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           will be correctly located with respect to either    the
           original    _f_l_e_x input file    (if the    errors are due to code
           in the input file), or lllleeeexxxx....yyyyyyyy....cccc (if the errors are
           _f_l_e_x'_s fault -- you should report these sorts of    errors
           to the email address given below).

      ----TTTT   makes _f_l_e_x run in _t_r_a_c_e mode.  It will generate a lot
           of messages to _s_t_d_e_r_r concerning    the form of the    input
           and the resultant non-deterministic and deterministic
           finite automata.     This option is    mostly for use in
           maintaining _f_l_e_x.

      ----VVVV   prints the version number to _s_t_d_o_u_t and exits.
           --------vvvveeeerrrrssssiiiioooonnnn is a synonym for ----VVVV....

      ----7777   instructs _f_l_e_x to generate a 7-bit scanner, i.e., one
           which can only recognized 7-bit characters in its
           input.  The advantage of    using ----7777 is that the scanner's
           tables can be up    to half    the size of those generated
           using the ----8888 option (see    below).     The disadvantage is
           that such scanners often    hang or    crash if their input
           contains    an 8-bit character.

           Note, however, that unless you generate your scanner
           using the ----CCCCffff or    ----CCCCFFFF table compression options, use of
           ----7777 will save only a small amount    of table space,    and
           make your scanner considerably less portable.  _F_l_e_x'_s
           default behavior    is to generate an 8-bit    scanner    unless
           you use the ----CCCCffff or ----CCCCFFFF,,,, in which    case _f_l_e_x defaults to
           generating 7-bit    scanners unless    your site was always
           configured to generate 8-bit scanners (as will often be
           the case    with non-USA sites).  You can tell whether
           flex generated a    7-bit or an 8-bit scanner by
           inspecting the flag summary in the ----vvvv output as
           described above.

           Note that if you    use ----CCCCffffeeee or ----CCCCFFFFeeee (those    table
           compression options, but    also using equivalence classes
           as discussed see    below),    flex still defaults to
           generating an 8-bit scanner, since usually with these
           compression options full    8-bit tables are not much more
           expensive than 7-bit tables.

      ----8888   instructs _f_l_e_x to generate an 8-bit scanner, i.e., one
           which can recognize 8-bit characters.  This flag    is
           only needed for scanners    generated using    ----CCCCffff or ----CCCCFFFF,,,, as
           otherwise flex defaults to generating an    8-bit scanner
           anyway.

           See the discussion of ----7777    above for flex's default
           behavior    and the    tradeoffs between 7-bit    and 8-bit
           scanners.



     Page 33                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      ----++++   specifies that you want flex to generate    a C++ scanner
           class.  See the section on Generating C++ Scanners
           below for details.

      ----CCCC[[[[aaaaeeeeffffFFFFmmmmrrrr]]]]
           controls    the degree of table compression    and, more
           generally, trade-offs between small scanners and    fast
           scanners.

           ----CCCCaaaa ("align") instructs flex to trade off larger    tables
           in the generated    scanner    for faster performance because
           the elements of the tables are better aligned for
           memory access and computation.  On some RISC
           architectures, fetching and manipulating    longwords is
           more efficient than with    smaller-sized units such as
           shortwords.  This option    can double the size of the
           tables used by your scanner.

           ----CCCCeeee directs _f_l_e_x    to construct _e_q_u_i_v_a_l_e_n_c_e _c_l_a_s_s_e_s,
           i.e., sets of characters    which have identical lexical
           properties (for example,    if the only appearance of
           digits in the _f_l_e_x input    is in the character class
           "[0-9]" then the    digits '0', '1', ..., '9' will all be
           put in the same equivalence class).  Equivalence
           classes usually give dramatic reductions    in the final
           table/object file sizes (typically a factor of 2-5) and
           are pretty cheap    performance-wise (one array look-up
           per character scanned).

           ----CCCCffff specifies that the _f_u_l_l scanner tables should be
           generated - _f_l_e_x    should not compress the    tables by
           taking advantages of similar transition functions for
           different states.

           ----CCCCFFFF specifies that the alternate    fast scanner
           representation (described above under the ----FFFF flag)
           should be used.    This option cannot be used with    ----++++....

           ----CCCCmmmm directs _f_l_e_x    to construct _m_e_t_a-_e_q_u_i_v_a_l_e_n_c_e _c_l_a_s_s_e_s,
           which are sets of equivalence classes (or characters,
           if equivalence classes are not being used) that are
           commonly    used together.    Meta-equivalence classes are
           often a big win when using compressed tables, but they
           have a moderate performance impact (one or two "if"
           tests and one array look-up per character scanned).

           ----CCCCrrrr causes the generated    scanner    to _b_y_p_a_s_s use of the
           standard    I/O library (stdio) for    input.    Instead    of
           calling ffffrrrreeeeaaaadddd(((()))) or ggggeeeettttcccc(((()))),,,, the scanner will use the
           rrrreeeeaaaadddd(((()))) system call, resulting in    a performance gain
           which varies from system    to system, but in general is
           probably    negligible unless you are also using ----CCCCffff or



     Page 34                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           ----CCCCFFFF.... Using ----CCCCrrrr can cause    strange    behavior if, for
           example,    you read from _y_y_i_n using stdio prior to
           calling the scanner (because the    scanner    will miss
           whatever    text your previous reads left in the stdio
           input buffer).

           ----CCCCrrrr has no effect if you    define YYYYYYYY____IIIINNNNPPPPUUUUTTTT    (see The
           Generated Scanner above).

           A lone ----CCCC specifies that    the scanner tables should be
           compressed but neither equivalence classes nor meta-
           equivalence classes should be used.

           The options ----CCCCffff or ----CCCCFFFF and ----CCCCmmmm do not make sense
           together    - there    is no opportunity for meta-equivalence
           classes if the table is not being compressed.
           Otherwise the options may be freely mixed, and are
           cumulative.

           The default setting is ----CCCCeeeemmmm,,,, which specifies that _f_l_e_x
           should generate equivalence classes and meta-
           equivalence classes.  This setting provides the highest
           degree of table compression.  You can trade off
           faster-executing    scanners at the    cost of    larger tables
           with the    following generally being true:

           slowest & smallest
             -Cem
             -Cm
             -Ce
             -C
             -C{f,F}e
             -C{f,F}
             -C{f,F}a
           fastest & largest

           Note that scanners with the smallest tables are usually
           generated and compiled the quickest, so during
           development you will usually want to use    the default,
           maximal compression.

           ----CCCCffffeeee is often a good compromise between speed and size
           for production scanners.

      ----oooooooouuuuttttppppuuuutttt
           directs flex to write the scanner to the    file oooouuuuttttppppuuuutttt
           instead of lllleeeexxxx....yyyyyyyy....cccc.... If you combine ----oooo with the ----tttt
           option, then the    scanner    is written to _s_t_d_o_u_t but its
           ####lllliiiinnnneeee directives    (see the ----LLLL option above) refer    to the
           file oooouuuuttttppppuuuutttt....

      ----PPPPpppprrrreeeeffffiiiixxxx



     Page 35                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           changes the default _y_y prefix used by _f_l_e_x for all
           globally-visible    variable and function names to instead
           be _p_r_e_f_i_x. For example, ----PPPPffffoooooooo changes the name of
           yyyyyyyytttteeeexxxxtttt to ffffooooooootttteeeexxxxtttt.... It also changes the name of the
           default output file from    lllleeeexxxx....yyyyyyyy....cccc to lllleeeexxxx....ffffoooooooo....cccc.... Here
           are all of the names affected:

           yy_create_buffer
           yy_delete_buffer
           yy_flex_debug
           yy_init_buffer
           yy_flush_buffer
           yy_load_buffer_state
           yy_switch_to_buffer
           yyin
           yyleng
           yylex
           yylineno
           yyout
           yyrestart
           yytext
           yywrap

           (If you are using a C++ scanner,    then only yyyyyyyywwwwrrrraaaapppp and
           yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr are affected.)  Within your scanner itself,
           you can still refer to the global variables and
           functions using either version of their name; but
           externally, they    have the modified name.

           This option lets    you easily link    together multiple _f_l_e_x
           programs    into the same executable.  Note, though, that
           using this option also renames yyyyyyyywwwwrrrraaaapppp(((()))),,,,    so you now
           _m_u_s_t either provide your    own (appropriately-named)
           version of the routine for your scanner,    or use %%%%ooooppppttttiiiioooonnnn
           nnnnooooyyyyyyyywwwwrrrraaaapppp,,,, as linking with ----llllffffllll no longer    provides one
           for you by default.

      ----SSSSsssskkkkeeeelllleeeettttoooonnnn____ffffiiiilllleeee
           overrides the default skeleton file from    which _f_l_e_x
           constructs its scanners.     You'll    never need this    option
           unless you are doing _f_l_e_x maintenance or    development.

      _f_l_e_x also provides a mechanism for controlling options
      within the scanner specification itself, rather than from
      the flex command-line.  This is done by including %%%%ooooppppttttiiiioooonnnn
      directives in    the first section of the scanner
      specification.  You can specify multiple options with    a
      single %%%%ooooppppttttiiiioooonnnn directive, and    multiple directives in the
      first    section    of your    flex input file.

      Most options are given simply    as names, optionally preceded
      by the word "no" (with no intervening    whitespace) to negate



     Page 36                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      their    meaning.  A number are equivalent to flex flags    or
      their    negation:

          7bit          -7 option
          8bit          -8 option
          align          -Ca option
          backup          -b option
          batch          -B option
          c++          -+ option

          caseful or
          case-sensitive  opposite of -i (default)

          case-insensitive or
          caseless          -i option

          debug          -d option
          default          opposite of -s option
          ecs          -Ce option
          fast          -F option
          full          -f option
          interactive     -I option
          lex-compat      -l option
          meta-ecs          -Cm option
          perf-report     -p option
          read          -Cr option
          stdout          -t option
          verbose          -v option
          warn          opposite of -w option
                  (use "%option nowarn" for    -w)

          array          equivalent to "%array"
          pointer          equivalent to "%pointer" (default)

      Some %%%%ooooppppttttiiiioooonnnn''''ssss provide features otherwise not    available:

      aaaallllwwwwaaaayyyyssss----iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee
           instructs flex to generate a scanner which always
           considers its input "interactive".  Normally, on    each
           new input file the scanner calls    iiiissssaaaattttttttyyyy(((()))) in an attempt
           to determine whether the    scanner's input    source is
           interactive and thus should be read a character at a
           time.  When this    option is used,    however, then no such
           call is made.

      mmmmaaaaiiiinnnn directs flex to provide a default mmmmaaaaiiiinnnn(((())))    program    for
           the scanner, which simply calls yyyyyyyylllleeeexxxx(((())))....    This option
           implies nnnnooooyyyyyyyywwwwrrrraaaapppp    (see below).

      nnnneeeevvvveeeerrrr----iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee
           instructs flex to generate a scanner which never
           considers its input "interactive" (again, no call made



     Page 37                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           to iiiissssaaaattttttttyyyy(((()))))))).... This is the opposite of aaaallllwwwwaaaayyyyssss----
           iiiinnnntttteeeerrrraaaaccccttttiiiivvvveeee....

      ssssttttaaaacccckkkk
           enables the use of start    condition stacks (see Start
           Conditions above).

      ssssttttddddiiiinnnniiiitttt
           if set (i.e., %%%%ooooppppttttiiiioooonnnn ssssttttddddiiiinnnniiiitttt)))) initializes _y_y_i_n and
           _y_y_o_u_t to    _s_t_d_i_n and _s_t_d_o_u_t, instead of the default of
           _n_i_l. Some existing _l_e_x programs depend on this
           behavior, even though it    is not compliant with ANSI C,
           which does not require _s_t_d_i_n and    _s_t_d_o_u_t to be compile-
           time constant.

      yyyyyyyylllliiiinnnneeeennnnoooo
           directs _f_l_e_x to generate    a scanner that maintains the
           number of the current line read from its    input in the
           global variable yyyyyyyylllliiiinnnneeeennnnoooo.... This option is    implied    by
           %%%%ooooppppttttiiiioooonnnn lllleeeexxxx----ccccoooommmmppppaaaatttt....

      yyyyyyyywwwwrrrraaaapppp
           if unset    (i.e., %%%%ooooppppttttiiiioooonnnn nnnnooooyyyyyyyywwwwrrrraaaapppp)))),,,, makes    the scanner
           not call    yyyyyyyywwwwrrrraaaapppp(((()))) upon an end-of-file, but simply
           assume that there are no    more files to scan (until the
           user points _y_y_i_n    at a new file and calls    yyyyyyyylllleeeexxxx(((())))
           again).

      _f_l_e_x scans your rule actions to determine whether you    use
      the RRRREEEEJJJJEEEECCCCTTTT or    yyyyyyyymmmmoooorrrreeee(((()))) features.  The    rrrreeeejjjjeeeecccctttt and yyyyyyyymmmmoooorrrreeee
      options are available    to override its    decision as to whether
      you use the options, either by setting them (e.g., %%%%ooooppppttttiiiioooonnnn
      rrrreeeejjjjeeeecccctttt)))) to indicate the feature is indeed used, or unsetting
      them to indicate it actually is not used (e.g., %%%%ooooppppttttiiiioooonnnn
      nnnnooooyyyyyyyymmmmoooorrrreeee))))....

      Three    options    take string-delimited values, offset with '=':

          %option outfile="ABC"

      is equivalent    to ----ooooAAAABBBBCCCC,,,, and

          %option prefix="XYZ"

      is equivalent    to ----PPPPXXXXYYYYZZZZ.... Finally,

          %option yyclass="foo"

      only applies when generating a C++ scanner ( ----++++ option).  It
      informs _f_l_e_x that you    have derived ffffoooooooo as a subclass of
      yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr,,,, so _f_l_e_x will place your actions in the member
      function ffffoooooooo::::::::yyyyyyyylllleeeexxxx(((())))    instead    of yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr::::::::yyyyyyyylllleeeexxxx(((()))).... It



     Page 38                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      also generates a yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr::::::::yyyyyyyylllleeeexxxx(((())))    member function    that
      emits    a run-time error (by invoking
      yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr::::::::LLLLeeeexxxxeeeerrrrEEEErrrrrrrroooorrrr(((()))))))) if    called.     See Generating    C++
      Scanners, below, for additional information.

      A number of options are available for    lint purists who want
      to suppress the appearance of    unneeded routines in the
      generated scanner.  Each of the following, if    unset (e.g.,
      %%%%ooooppppttttiiiioooonnnn nnnnoooouuuunnnnppppuuuutttt ), results in    the corresponding routine not
      appearing in the generated scanner:

          input, unput
          yy_push_state, yy_pop_state, yy_top_state
          yy_scan_buffer, yy_scan_bytes, yy_scan_string

      (though yyyyyyyy____ppppuuuusssshhhh____ssssttttaaaatttteeee(((()))) and friends won't appear anyway
      unless you use %%%%ooooppppttttiiiioooonnnn ssssttttaaaacccckkkk))))....

     PPPPEEEERRRRFFFFOOOORRRRMMMMAAAANNNNCCCCEEEE CCCCOOOONNNNSSSSIIIIDDDDEEEERRRRAAAATTTTIIIIOOOONNNNSSSS
      The main design goal of _f_l_e_x is that it generate high-
      performance scanners.     It has    been optimized for dealing
      well with large sets of rules.  Aside    from the effects on
      scanner speed    of the table compression ----CCCC options outlined
      above, there are a number of options/actions which degrade
      performance.    These are, from    most expensive to least:

          REJECT
          %option yylineno
          arbitrary    trailing context

          pattern sets that    require    backing    up
          %array
          %option interactive
          %option always-interactive

          '^' beginning-of-line operator
          yymore()

      with the first three all being quite expensive and the last
      two being quite cheap.  Note also that uuuunnnnppppuuuutttt(((()))) is
      implemented as a routine call    that potentially does quite a
      bit of work, while yyyyyyyylllleeeessssssss(((()))) is a quite-cheap macro; so if
      just putting back some excess    text you scanned, use
      yyyyyyyylllleeeessssssss(((())))....

      RRRREEEEJJJJEEEECCCCTTTT should    be avoided at all costs    when performance is
      important.  It is a particularly expensive option.

      Getting rid of backing up is messy and often may be an
      enormous amount of work for a    complicated scanner.  In
      principal, one begins    by using the ----bbbb    flag to    generate a
      _l_e_x._b_a_c_k_u_p file.  For    example, on the    input



     Page 39                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          %%
          foo     return    TOK_KEYWORD;
          foobar     return    TOK_KEYWORD;

      the file looks like:

          State #6 is non-accepting    -
           associated rule line numbers:
             2         3
           out-transitions:    [ o ]
           jam-transitions:    EOF [ \001-n  p-\177 ]

          State #8 is non-accepting    -
           associated rule line numbers:
             3
           out-transitions:    [ a ]
           jam-transitions:    EOF [ \001-`  b-\177 ]

          State #9 is non-accepting    -
           associated rule line numbers:
             3
           out-transitions:    [ r ]
           jam-transitions:    EOF [ \001-q  s-\177 ]

          Compressed tables    always back up.

      The first few    lines tell us that there's a scanner state in
      which    it can make a transition on an 'o' but not on any
      other    character, and that in that state the currently
      scanned text does not    match any rule.     The state occurs when
      trying to match the rules found at lines 2 and 3 in the
      input    file.  If the scanner is in that state and then    reads
      something other than an 'o', it will have to back up to find
      a rule which is matched.  With a bit of headscratching one
      can see that this must be the    state it's in when it has seen
      "fo".     When this has happened, if anything other than
      another 'o' is seen, the scanner will    have to    back up    to
      simply match the 'f' (by the default rule).

      The comment regarding    State #8 indicates there's a problem
      when "foob" has been scanned.     Indeed, on any    character
      other    than an    'a', the scanner will have to back up to
      accept "foo".     Similarly, the    comment    for State #9 concerns
      when "fooba" has been    scanned    and an 'r' does    not follow.

      The final comment reminds us that there's no point going to
      all the trouble of removing backing up from the rules    unless
      we're    using ----CCCCffff or ----CCCCFFFF,,,, since    there's    no performance gain
      doing    so with    compressed scanners.

      The way to remove the    backing    up is to add "error" rules:




     Page 40                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          %%
          foo      return TOK_KEYWORD;
          foobar      return TOK_KEYWORD;

          fooba      |
          foob      |
          fo      {
              /* false alarm, not really a keyword */
              return TOK_ID;
              }


      Eliminating backing up among a list of keywords can also be
      done using a "catch-all" rule:

          %%
          foo      return TOK_KEYWORD;
          foobar      return TOK_KEYWORD;

          [a-z]+      return TOK_ID;

      This is usually the best solution when appropriate.

      Backing up messages tend to cascade.    With a complicated set
      of rules it's    not uncommon to    get hundreds of    messages.  If
      one can decipher them, though, it often only takes a dozen
      or so    rules to eliminate the backing up (though it's easy to
      make a mistake and have an error rule    accidentally match a
      valid    token.    A possible future _f_l_e_x feature will be to
      automatically    add rules to eliminate backing up).

      It's important to keep in mind that you gain the benefits of
      eliminating backing up only if you eliminate _e_v_e_r_y instance
      of backing up.  Leaving just one means you gain nothing.

      _V_a_r_i_a_b_l_e trailing context (where both    the leading and
      trailing parts do not    have a fixed length) entails almost
      the same performance loss as RRRREEEEJJJJEEEECCCCTTTT (i.e., substantial).  So
      when possible    a rule like:

          %%
          mouse|rat/(cat|dog)   run();

      is better written:

          %%
          mouse/cat|dog        run();
          rat/cat|dog        run();

      or as

          %%



     Page 41                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          mouse|rat/cat        run();
          mouse|rat/dog        run();

      Note that here the special '|' action    does _n_o_t provide any
      savings, and can even    make things worse (see Deficiencies /
      Bugs below).

      Another area where the user can increase a scanner's
      performance (and one that's easier to    implement) arises from
      the fact that    the longer the tokens matched, the faster the
      scanner will run.  This is because with long tokens the
      processing of    most input characters takes place in the
      (short) inner    scanning loop, and does    not often have to go
      through the additional work of setting up the    scanning
      environment (e.g., yyyyyyyytttteeeexxxxtttt)))) for the action.  Recall the
      scanner for C    comments:

          %x comment
          %%
              int line_num = 1;

          "/*"       BEGIN(comment);

          <comment>[^*\n]*
          <comment>"*"+[^*/\n]*
          <comment>\n          ++line_num;
          <comment>"*"+"/"          BEGIN(INITIAL);

      This could be    sped up    by writing it as:

          %x comment
          %%
              int line_num = 1;

          "/*"       BEGIN(comment);

          <comment>[^*\n]*
          <comment>[^*\n]*\n      ++line_num;
          <comment>"*"+[^*/\n]*
          <comment>"*"+[^*/\n]*\n ++line_num;
          <comment>"*"+"/"          BEGIN(INITIAL);

      Now instead of each newline requiring    the processing of
      another action, recognizing the newlines is "distributed"
      over the other rules to keep the matched text    as long    as
      possible.  Note that _a_d_d_i_n_g rules does _n_o_t slow down the
      scanner!  The    speed of the scanner is    independent of the
      number of rules or (modulo the considerations    given at the
      beginning of this section) how complicated the rules are
      with regard to operators such    as '*' and '|'.

      A final example in speeding up a scanner: suppose you    want



     Page 42                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      to scan through a file containing identifiers    and keywords,
      one per line and with    no other extraneous characters,    and
      recognize all    the keywords.  A natural first approach    is:

          %%
          asm      |
          auto     |
          break    |
          ... etc ...
          volatile |
          while    /* it's a keyword */

          .|\n     /* it's not a keyword */

      To eliminate the back-tracking, introduce a catch-all    rule:

          %%
          asm      |
          auto     |
          break    |
          ... etc ...
          volatile |
          while    /* it's a keyword */

          [a-z]+   |
          .|\n     /* it's not a keyword */

      Now, if it's guaranteed that there's exactly one word    per
      line,    then we    can reduce the total number of matches by a
      half by merging in the recognition of    newlines with that of
      the other tokens:

          %%
          asm\n    |
          auto\n   |
          break\n  |
          ... etc ...
          volatile\n |
          while\n  /* it's a keyword */

          [a-z]+\n |
          .|\n     /* it's not a keyword */

      One has to be    careful    here, as we have now reintroduced
      backing up into the scanner.    In particular, while _w_e    know
      that there will never    be any characters in the input stream
      other    than letters or    newlines, _f_l_e_x can't figure this out,
      and it will plan for possibly    needing    to back    up when    it has
      scanned a token like "auto" and then the next    character is
      something other than a newline or a letter.  Previously it
      would    then just match    the "auto" rule    and be done, but now
      it has no "auto" rule, only a    "auto\n" rule.    To eliminate



     Page 43                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      the possibility of backing up, we could either duplicate all
      rules    but without final newlines, or,    since we never expect
      to encounter such an input and therefore don't how it's
      classified, we can introduce one more    catch-all rule,    this
      one which doesn't include a newline:

          %%
          asm\n    |
          auto\n   |
          break\n  |
          ... etc ...
          volatile\n |
          while\n  /* it's a keyword */

          [a-z]+\n |
          [a-z]+   |
          .|\n     /* it's not a keyword */

      Compiled with    ----CCCCffff,,,, this is about as fast as one can get a
      _f_l_e_x scanner to go for this particular problem.

      A final note:     _f_l_e_x is slow when matching NUL's,
      particularly when a token contains multiple NUL's.  It's
      best to write    rules which match _s_h_o_r_t    amounts    of text    if
      it's anticipated that    the text will often include NUL's.

      Another final    note regarding performance: as mentioned above
      in the section How the Input is Matched, dynamically
      resizing yyyyyyyytttteeeexxxxtttt to accommodate huge tokens is    a slow process
      because it presently requires    that the (huge)    token be
      rescanned from the beginning.     Thus if performance is    vital,
      you should attempt to    match "large" quantities of text but
      not "huge" quantities, where the cutoff between the two is
      at about 8K characters/token.

     GGGGEEEENNNNEEEERRRRAAAATTTTIIIINNNNGGGG    CCCC++++++++ SSSSCCCCAAAANNNNNNNNEEEERRRRSSSS
      _f_l_e_x provides    two different ways to generate scanners    for
      use with C++.     The first way is to simply compile a scanner
      generated by _f_l_e_x using a C++    compiler instead of a C
      compiler.  You should    not encounter any compilations errors
      (please report any you find to the email address given in
      the Author section below).  You can then use C++ code    in
      your rule actions instead of C code.    Note that the default
      input    source for your    scanner    remains    _y_y_i_n, and default
      echoing is still done    to _y_y_o_u_t. Both of these    remain _F_I_L_E *
      variables and    not C++    _s_t_r_e_a_m_s.

      You can also use _f_l_e_x    to generate a C++ scanner class, using
      the ----++++ option    (or, equivalently, %%%%ooooppppttttiiiioooonnnn cccc++++++++)))),,,, which is
      automatically    specified if the name of the flex executable
      ends in a '+', such as _f_l_e_x++. When using this option, flex
      defaults to generating the scanner to    the file lllleeeexxxx....yyyyyyyy....cccccccc



     Page 44                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      instead of lllleeeexxxx....yyyyyyyy....cccc.... The generated scanner includes the
      header file _F_l_e_x_L_e_x_e_r._h, which defines the interface to two
      C++ classes.

      The first class, FFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr,,,, provides an abstract base    class
      defining the general scanner class interface.     It provides
      the following    member functions:

      ccccoooonnnnsssstttt    cccchhhhaaaarrrr**** YYYYYYYYTTTTeeeexxxxtttt(((())))
           returns the text    of the most recently matched token,
           the equivalent of yyyyyyyytttteeeexxxxtttt....

      iiiinnnntttt YYYYYYYYLLLLeeeennnngggg(((())))
           returns the length of the most recently matched token,
           the equivalent of yyyyyyyylllleeeennnngggg....

      iiiinnnntttt lllliiiinnnneeeennnnoooo(((()))) ccccoooonnnnsssstttt
           returns the current input line number (see %%%%ooooppppttttiiiioooonnnn
           yyyyyyyylllliiiinnnneeeennnnoooo)))),,,, or 1111 if %%%%ooooppppttttiiiioooonnnn yyyyyyyylllliiiinnnneeeennnnoooo was not used.

      vvvvooooiiiidddd sssseeeetttt____ddddeeeebbbbuuuugggg(((( iiiinnnntttt ffffllllaaaagggg ))))
           sets the    debugging flag for the scanner,    equivalent to
           assigning to yyyyyyyy____fffflllleeeexxxx____ddddeeeebbbbuuuugggg (see the Options section
           above).    Note that you must build the scanner using
           %%%%ooooppppttttiiiioooonnnn ddddeeeebbbbuuuugggg to    include    debugging information in it.

      iiiinnnntttt ddddeeeebbbbuuuugggg(((()))) ccccoooonnnnsssstttt
           returns the current setting of the debugging flag.

      Also provided    are member functions equivalent    to
      yyyyyyyy____sssswwwwiiiittttcccchhhh____ttttoooo____bbbbuuuuffffffffeeeerrrr(((()))),,,, yyyyyyyy____ccccrrrreeeeaaaatttteeee____bbbbuuuuffffffffeeeerrrr(((()))) (though the    first
      argument is an iiiissssttttrrrreeeeaaaammmm**** object pointer and not a FFFFIIIILLLLEEEE****)))),,,,
      yyyyyyyy____fffflllluuuusssshhhh____bbbbuuuuffffffffeeeerrrr(((()))),,,, yyyyyyyy____ddddeeeelllleeeetttteeee____bbbbuuuuffffffffeeeerrrr(((()))),,,, and yyyyyyyyrrrreeeessssttttaaaarrrrtttt(((())))
      (again, the first argument is    a iiiissssttttrrrreeeeaaaammmm**** object pointer).

      The second class defined in _F_l_e_x_L_e_x_e_r._h is yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr,,,,
      which    is derived from    FFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr.... It defines the following
      additional member functions:

      yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr(((( iiiissssttttrrrreeeeaaaammmm****    aaaarrrrgggg____yyyyyyyyiiiinnnn ==== 0000,,,, oooossssttttrrrreeeeaaaammmm**** aaaarrrrgggg____yyyyyyyyoooouuuutttt ==== 0000 ))))
           constructs a yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr    object using the given streams
           for input and output.  If not specified,    the streams
           default to cccciiiinnnn and ccccoooouuuutttt,,,,    respectively.

      vvvviiiirrrrttttuuuuaaaallll iiiinnnntttt yyyyyyyylllleeeexxxx(((())))
           performs    the same role is yyyyyyyylllleeeexxxx(((()))) does for ordinary
           flex scanners: it scans the input stream, consuming
           tokens, until a rule's action returns a value.  If you
           derive a    subclass SSSS from    yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr and    want to    access
           the member functions and    variables of SSSS inside yyyyyyyylllleeeexxxx(((()))),,,,
           then you    need to    use %%%%ooooppppttttiiiioooonnnn yyyyyyyyccccllllaaaassssssss====""""SSSS""""    to inform _f_l_e_x
           that you    will be    using that subclass instead of



     Page 45                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr.... In this case, rather than generating
           yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr::::::::yyyyyyyylllleeeexxxx(((()))),,,, _f_l_e_x generates SSSS::::::::yyyyyyyylllleeeexxxx(((()))) (and
           also generates a    dummy yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr::::::::yyyyyyyylllleeeexxxx(((()))) that    calls
           yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr::::::::LLLLeeeexxxxeeeerrrrEEEErrrrrrrroooorrrr(((()))) if called).

      vvvviiiirrrrttttuuuuaaaallll vvvvooooiiiidddd sssswwwwiiiittttcccchhhh____ssssttttrrrreeeeaaaammmmssss((((iiiissssttttrrrreeeeaaaammmm**** nnnneeeewwww____iiiinnnn ====    0000,,,,
           oooossssttttrrrreeeeaaaammmm****    nnnneeeewwww____oooouuuutttt    ==== 0000)))) reassigns yyyyyyyyiiiinnnn to nnnneeeewwww____iiiinnnn (if
           non-nil)    and yyyyyyyyoooouuuutttt to nnnneeeewwww____oooouuuutttt (ditto), deleting the
           previous    input buffer if    yyyyyyyyiiiinnnn is    reassigned.

      iiiinnnntttt yyyyyyyylllleeeexxxx(((( iiiissssttttrrrreeeeaaaammmm**** nnnneeeewwww____iiiinnnn,,,, oooossssttttrrrreeeeaaaammmm**** nnnneeeewwww____oooouuuutttt ==== 0000 ))))
           first switches the input    streams    via sssswwwwiiiittttcccchhhh____ssssttttrrrreeeeaaaammmmssss((((
           nnnneeeewwww____iiiinnnn,,,, nnnneeeewwww____oooouuuutttt )))) and then returns the value of
           yyyyyyyylllleeeexxxx(((())))....

      In addition, yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr defines the following protected
      virtual functions which you can redefine in derived classes
      to tailor the    scanner:

      vvvviiiirrrrttttuuuuaaaallll iiiinnnntttt LLLLeeeexxxxeeeerrrrIIIInnnnppppuuuutttt(((( cccchhhhaaaarrrr****    bbbbuuuuffff,,,, iiiinnnntttt mmmmaaaaxxxx____ssssiiiizzzzeeee ))))
           reads up    to mmmmaaaaxxxx____ssssiiiizzzzeeee characters into bbbbuuuuffff    and returns
           the number of characters    read.  To indicate end-of-
           input, return 0 characters.  Note that "interactive"
           scanners    (see the ----BBBB and    ----IIII flags) define the macro
           YYYYYYYY____IIIINNNNTTTTEEEERRRRAAAACCCCTTTTIIIIVVVVEEEE.... If you redefine LLLLeeeexxxxeeeerrrrIIIInnnnppppuuuutttt(((()))) and    need
           to take different actions depending on whether or not
           the scanner might be scanning an    interactive input
           source, you can test for    the presence of    this name via
           ####iiiiffffddddeeeeffff....

      vvvviiiirrrrttttuuuuaaaallll vvvvooooiiiidddd LLLLeeeexxxxeeeerrrrOOOOuuuuttttppppuuuutttt(((( ccccoooonnnnsssstttt cccchhhhaaaarrrr****    bbbbuuuuffff,,,, iiiinnnntttt ssssiiiizzzzeeee ))))
           writes out ssssiiiizzzzeeee characters from the buffer bbbbuuuuffff,,,, which,
           while NUL-terminated, may also contain "internal" NUL's
           if the scanner's    rules can match    text with NUL's    in
           them.

      vvvviiiirrrrttttuuuuaaaallll vvvvooooiiiidddd LLLLeeeexxxxeeeerrrrEEEErrrrrrrroooorrrr(((( ccccoooonnnnsssstttt cccchhhhaaaarrrr**** mmmmssssgggg ))))
           reports a fatal error message.  The default version of
           this function writes the    message    to the stream cccceeeerrrrrrrr and
           exits.

      Note that a yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr object contains its _e_n_t_i_r_e scanning
      state.  Thus you can use such    objects    to create reentrant
      scanners.  You can instantiate multiple instances of the
      same yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr class, and you can also combine multiple
      C++ scanner classes together in the same program using the
      ----PPPP option discussed above.

      Finally, note    that the %%%%aaaarrrrrrrraaaayyyy    feature    is not available to
      C++ scanner classes; you must    use %%%%ppppooooiiiinnnntttteeeerrrr (the default).

      Here is an example of    a simple C++ scanner:



     Page 46                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          // An    example    of using the flex C++ scanner class.

          %{
          int mylineno = 0;
          %}

          string  \"[^\n"]+\"

          ws      [    \t]+

          alpha   [A-Za-z]
          dig     [0-9]
          name    ({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])*
          num1    [-+]?{dig}+\.?([eE][-+]?{dig}+)?
          num2    [-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)?
          number  {num1}|{num2}

          %%

          {ws}    /* skip blanks and tabs */

          "/*"    {
              int c;

              while((c = yyinput()) != 0)
              {
              if(c == '\n')
                  ++mylineno;

              else if(c == '*')
                  {
                  if((c = yyinput()) == '/')
                  break;
                  else
                  unput(c);
                  }
              }
              }

          {number}    cout <<    "number    " << YYText() << '\n';

          \n    mylineno++;

          {name}    cout <<    "name "    << YYText() << '\n';

          {string}    cout <<    "string    " << YYText() << '\n';

          %%

          int main(    int /* argc */,    char** /* argv */ )
          {
          FlexLexer* lexer = new yyFlexLexer;



     Page 47                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



          while(lexer->yylex() != 0)
              ;
          return 0;
          }
      If you want to create    multiple (different) lexer classes,
      you use the ----PPPP flag (or the pppprrrreeeeffffiiiixxxx==== option) to rename    each
      yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr to some other xxxxxxxxFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr.... You then can include
      <<<<FFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr....hhhh>>>>    in your    other sources once per lexer class,
      first    renaming yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr as    follows:

          #undef yyFlexLexer
          #define yyFlexLexer xxFlexLexer
          #include <FlexLexer.h>

          #undef yyFlexLexer
          #define yyFlexLexer zzFlexLexer
          #include <FlexLexer.h>

      if, for example, you used %%%%ooooppppttttiiiioooonnnn pppprrrreeeeffffiiiixxxx====""""xxxxxxxx""""    for one    of
      your scanners    and %%%%ooooppppttttiiiioooonnnn pppprrrreeeeffffiiiixxxx====""""zzzzzzzz""""    for the    other.

      IMPORTANT: the present form of the scanning class is
      _e_x_p_e_r_i_m_e_n_t_a_l and may change considerably between major
      releases.

     IIIINNNNCCCCOOOOMMMMPPPPAAAATTTTIIIIBBBBIIIILLLLIIIITTTTIIIIEEEESSSS WWWWIIIITTTTHHHH LLLLEEEEXXXX    AAAANNNNDDDD PPPPOOOOSSSSIIIIXXXX
      _f_l_e_x is a rewrite of the AT&T    Unix _l_e_x tool (the two
      implementations do not share any code, though), with some
      extensions and incompatibilities, both of which are of
      concern to those who wish to write scanners acceptable to
      either implementation.  Flex is fully    compliant with the
      POSIX    _l_e_x specification, except that when using %%%%ppppooooiiiinnnntttteeeerrrr
      (the default), a call    to uuuunnnnppppuuuutttt(((()))) destroys the    contents of
      yyyyyyyytttteeeexxxxtttt,,,, which    is counter to the POSIX    specification.

      In this section we discuss all of the    known areas of
      incompatibility between flex,    AT&T lex, and the POSIX
      specification.

      _f_l_e_x'_s ----llll option turns on maximum compatibility with the
      original AT&T    _l_e_x implementation, at the cost    of a major
      loss in the generated    scanner's performance.    We note    below
      which    incompatibilities can be overcome using    the ----llll option.

      _f_l_e_x is fully    compatible with    _l_e_x with the following
      exceptions:

      -    The undocumented    _l_e_x scanner internal variable yyyyyyyylllliiiinnnneeeennnnoooo
           is not supported    unless ----llll or %%%%ooooppppttttiiiioooonnnn yyyyyyyylllliiiinnnneeeennnnoooo is used.

           yyyyyyyylllliiiinnnneeeennnnoooo    should be maintained on    a per-buffer basis,
           rather than a per-scanner (single global    variable)



     Page 48                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           basis.

           yyyyyyyylllliiiinnnneeeennnnoooo    is not part of the POSIX specification.

      -    The iiiinnnnppppuuuutttt(((()))) routine is not redefinable, though it may
           be called to read characters following whatever has
           been matched by a rule.    If iiiinnnnppppuuuutttt(((()))) encounters an end-
           of-file the normal yyyyyyyywwwwrrrraaaapppp(((()))) processing is done.    A
           ``real''    end-of-file is returned    by iiiinnnnppppuuuutttt(((()))) as _E_O_F.

           Input is    instead    controlled by defining the YYYYYYYY____IIIINNNNPPPPUUUUTTTT
           macro.

           The _f_l_e_x    restriction that iiiinnnnppppuuuutttt(((()))) cannot    be redefined
           is in accordance    with the POSIX specification, which
           simply does not specify any way of controlling the
           scanner's input other than by making an initial
           assignment to _y_y_i_n.

      -    The uuuunnnnppppuuuutttt(((()))) routine is not redefinable.    This
           restriction is in accordance with POSIX.

      -    _f_l_e_x scanners are not as    reentrant as _l_e_x scanners.  In
           particular, if you have an interactive scanner and an
           interrupt handler which long-jumps out of the scanner,
           and the scanner is subsequently called again, you may
           get the following message:

           fatal flex scanner internal error--end of buffer missed

           To reenter the scanner, first use

           yyrestart( yyin );

           Note that this call will    throw away any buffered    input;
           usually this isn't a problem with an interactive
           scanner.

           Also note that flex C++ scanner classes _a_r_e reentrant,
           so if using C++ is an option for    you, you should    use
           them instead.  See "Generating C++ Scanners" above for
           details.

      -    oooouuuuttttppppuuuutttt(((())))    is not supported.  Output from the EEEECCCCHHHHOOOO    macro
           is done to the file-pointer _y_y_o_u_t (default _s_t_d_o_u_t).

           oooouuuuttttppppuuuutttt(((())))    is not part of the POSIX specification.

      -    _l_e_x does    not support exclusive start conditions (%x),
           though they are in the POSIX specification.

      -    When definitions    are expanded, _f_l_e_x encloses them in



     Page 49                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



           parentheses.  With lex, the following:

           NAME       [A-Z][A-Z0-9]*
           %%
           foo{NAME}?       printf( "Found it\n"    );
           %%

           will not    match the string "foo" because when the    macro
           is expanded the rule is equivalent to "foo[A-Z][A-Z0-
           9]*?"  and the precedence is such that the '?' is
           associated with "[A-Z0-9]*".  With _f_l_e_x,    the rule will
           be expanded to "foo([A-Z][A-Z0-9]*)?" and so the    string
           "foo" will match.

           Note that if the    definition begins with ^^^^ or ends with
           $$$$ then it is _n_o_t    expanded with parentheses, to allow
           these operators to appear in definitions    without    losing
           their special meanings.    But the    <<<<ssss>>>>,,,, ////,,,,    and <<<<<<<<EEEEOOOOFFFF>>>>>>>>
           operators cannot    be used    in a _f_l_e_x definition.

           Using ----llll    results    in the _l_e_x behavior of no parentheses
           around the definition.

           The POSIX specification is that the definition be
           enclosed    in parentheses.

      -    Some implementations of _l_e_x allow a rule's action to
           begin on    a separate line, if the    rule's pattern has
           trailing    whitespace:

           %%
           foo|bar<space here>
             { foobar_action();    }

           _f_l_e_x does not support this feature.

      -    The _l_e_x %%%%rrrr (generate a Ratfor scanner) option is    not
           supported.  It is not part of the POSIX specification.

      -    After a call to uuuunnnnppppuuuutttt(((()))),,,,    _y_y_t_e_x_t is undefined until the
           next token is matched, unless the scanner was built
           using %%%%aaaarrrrrrrraaaayyyy.... This is not the case with _l_e_x or the
           POSIX specification.  The ----llll option does    away with this
           incompatibility.

      -    The precedence of the {{{{}}}}    (numeric range)    operator is
           different.  _l_e_x interprets "abc{1,3}" as    "match one,
           two, or three occurrences of 'abc'", whereas _f_l_e_x
           interprets it as    "match 'ab' followed by    one, two, or
           three occurrences of 'c'".  The latter is in agreement
           with the    POSIX specification.




     Page 50                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      -    The precedence of the ^^^^ operator    is different.  _l_e_x
           interprets "^foo|bar" as    "match either 'foo' at the
           beginning of a line, or 'bar' anywhere",    whereas    _f_l_e_x
           interprets it as    "match either 'foo' or 'bar' if    they
           come at the beginning of    a line".  The latter is    in
           agreement with the POSIX    specification.

      -    The special table-size declarations such    as %%%%aaaa
           supported by _l_e_x    are not    required by _f_l_e_x scanners;
           _f_l_e_x ignores them.

      -    The name    FLEX_SCANNER is    #define'd so scanners may be
           written for use with either _f_l_e_x    or _l_e_x.    Scanners also
           include YYYYYYYY____FFFFLLLLEEEEXXXX____MMMMAAAAJJJJOOOORRRR____VVVVEEEERRRRSSSSIIIIOOOONNNN and YYYYYYYY____FFFFLLLLEEEEXXXX____MMMMIIIINNNNOOOORRRR____VVVVEEEERRRRSSSSIIIIOOOONNNN
           indicating which    version    of _f_l_e_x    generated the scanner
           (for example, for the 2.5 release, these    defines    would
           be 2 and    5 respectively).

      The following    _f_l_e_x features are not included in _l_e_x or the
      POSIX    specification:

          C++ scanners
          %option
          start condition scopes
          start condition stacks
          interactive/non-interactive scanners
          yy_scan_string() and friends
          yyterminate()
          yy_set_interactive()
          yy_set_bol()
          YY_AT_BOL()
          <<EOF>>
          <*>
          YY_DECL
          YY_START
          YY_USER_ACTION
          YY_USER_INIT
          #line directives
          %{}'s around actions
          multiple actions on a line

      plus almost all of the flex flags.  The last feature in the
      list refers to the fact that with _f_l_e_x you can put multiple
      actions on the same line, separated with semi-colons,    while
      with _l_e_x, the    following

          foo    handle_foo(); ++num_foos_seen;

      is (rather surprisingly) truncated to

          foo    handle_foo();




     Page 51                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      _f_l_e_x does not    truncate the action.  Actions that are not
      enclosed in braces are simply    terminated at the end of the
      line.

     DDDDIIIIAAAAGGGGNNNNOOOOSSSSTTTTIIIICCCCSSSS
      _w_a_r_n_i_n_g, _r_u_l_e    _c_a_n_n_o_t _b_e _m_a_t_c_h_e_d indicates that the given
      rule cannot be matched because it follows other rules    that
      will always match the    same text as it.  For example, in the
      following "foo" cannot be matched because it comes after an
      identifier "catch-all" rule:

          [a-z]+    got_identifier();
          foo    got_foo();

      Using    RRRREEEEJJJJEEEECCCCTTTT in a scanner suppresses this warning.

      _w_a_r_n_i_n_g, ----ssss _o_p_t_i_o_n _g_i_v_e_n _b_u_t _d_e_f_a_u_l_t _r_u_l_e _c_a_n    _b_e _m_a_t_c_h_e_d
      means    that it    is possible (perhaps only in a particular
      start    condition) that    the default rule (match    any single
      character) is    the only one that will match a particular
      input.  Since    ----ssss was given, presumably this is not intended.

      _r_e_j_e_c_t__u_s_e_d__b_u_t__n_o_t__d_e_t_e_c_t_e_d _u_n_d_e_f_i_n_e_d or
      _y_y_m_o_r_e__u_s_e_d__b_u_t__n_o_t__d_e_t_e_c_t_e_d _u_n_d_e_f_i_n_e_d - These errors    can
      occur    at compile time.  They indicate    that the scanner uses
      RRRREEEEJJJJEEEECCCCTTTT or yyyyyyyymmmmoooorrrreeee(((()))) but that _f_l_e_x failed to notice the    fact,
      meaning that _f_l_e_x scanned the    first two sections looking for
      occurrences of these actions and failed to find any, but
      somehow you snuck some in (via a #include file, for
      example).  Use %%%%ooooppppttttiiiioooonnnn rrrreeeejjjjeeeecccctttt    or %%%%ooooppppttttiiiioooonnnn yyyyyyyymmmmoooorrrreeee to indicate
      to flex that you really do use these features.

      _f_l_e_x _s_c_a_n_n_e_r _j_a_m_m_e_d -    a scanner compiled with    ----ssss has
      encountered an input string which wasn't matched by any of
      its rules.  This error can also occur    due to internal
      problems.

      _t_o_k_e_n    _t_o_o _l_a_r_g_e, _e_x_c_e_e_d_s _Y_Y_L_M_A_X - your scanner uses %%%%aaaarrrrrrrraaaayyyy
      and one of its rules matched a string    longer than the    YYYYYYYYLLLLMMMMAAAAXXXX
      constant (8K bytes by    default).  You can increase the    value
      by #define'ing YYYYYYYYLLLLMMMMAAAAXXXX    in the definitions section of your
      _f_l_e_x input.

      _s_c_a_n_n_e_r _r_e_q_u_i_r_e_s -_8 _f_l_a_g _t_o _u_s_e _t_h_e _c_h_a_r_a_c_t_e_r    '_x' - Your
      scanner specification    includes recognizing the 8-bit
      character '_x'    and you    did not    specify    the -8 flag, and your
      scanner defaulted to 7-bit because you used the ----CCCCffff or ----CCCCFFFF
      table    compression options.  See the discussion of the    ----7777
      flag for details.

      _f_l_e_x _s_c_a_n_n_e_r _p_u_s_h-_b_a_c_k _o_v_e_r_f_l_o_w - you    used uuuunnnnppppuuuutttt(((()))) to    push
      back so much text that the scanner's buffer could not    hold



     Page 52                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      both the pushed-back text and    the current token in yyyyyyyytttteeeexxxxtttt....
      Ideally the scanner should dynamically resize    the buffer in
      this case, but at present it does not.

      _i_n_p_u_t    _b_u_f_f_e_r _o_v_e_r_f_l_o_w, _c_a_n'_t _e_n_l_a_r_g_e _b_u_f_f_e_r _b_e_c_a_u_s_e _s_c_a_n_n_e_r
      _u_s_e_s _R_E_J_E_C_T -    the scanner was    working    on matching an
      extremely large token    and needed to expand the input buffer.
      This doesn't work with scanners that use RRRREEEEJJJJEEEECCCCTTTT....

      _f_a_t_a_l    _f_l_e_x _s_c_a_n_n_e_r _i_n_t_e_r_n_a_l _e_r_r_o_r--_e_n_d _o_f _b_u_f_f_e_r _m_i_s_s_e_d -
      This can occur in an scanner which is    reentered after    a
      long-jump has    jumped out (or over) the scanner's activation
      frame.  Before reentering the    scanner, use:

          yyrestart( yyin );

      or, as noted above, switch to    using the C++ scanner class.

      _t_o_o _m_a_n_y _s_t_a_r_t _c_o_n_d_i_t_i_o_n_s _i_n <> you listed more start
      conditions in    a <> construct than exist (so you must have
      listed at least one of them twice).

     FFFFIIIILLLLEEEESSSS
      ----llllffffllll library with which scanners must    be linked.

      _l_e_x._y_y._c
           generated scanner (called _l_e_x_y_y._c on some systems).

      _l_e_x._y_y._c_c
           generated C++ scanner class, when using ----++++....

      <_F_l_e_x_L_e_x_e_r._h>
           header file defining the    C++ scanner base class,
           FFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr,,,, and its derived class, yyyyyyyyFFFFlllleeeexxxxLLLLeeeexxxxeeeerrrr....

      _f_l_e_x._s_k_l
           skeleton    scanner.  This file is only used when building
           flex, not when flex executes.

      _l_e_x._b_a_c_k_u_p
           backing-up information for ----bbbb flag (called _l_e_x._b_c_k on
           some systems).

     DDDDEEEEFFFFIIIICCCCIIIIEEEENNNNCCCCIIIIEEEESSSS //// BBBBUUUUGGGGSSSS
      Some trailing    context    patterns cannot    be properly matched
      and generate warning messages    ("dangerous trailing
      context").  These are    patterns where the ending of the first
      part of the rule matches the beginning of the    second part,
      such as "zx*/xy*", where the 'x*' matches the    'x' at the
      beginning of the trailing context.  (Note that the POSIX
      draft    states that the    text matched by    such patterns is
      undefined.)



     Page 53                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      For some trailing context rules, parts which are actually
      fixed-length are not recognized as such, leading to the
      abovementioned performance loss.  In particular, parts using
      '|' or {n} (such as "foo{3}")    are always considered
      variable-length.

      Combining trailing context with the special '|' action can
      result in _f_i_x_e_d trailing context being turned    into the more
      expensive _v_a_r_i_a_b_l_e trailing context.    For example, in    the
      following:

          %%
          abc      |
          xyz/def


      Use of uuuunnnnppppuuuutttt(((()))) invalidates yytext and    yyleng,    unless the
      %%%%aaaarrrrrrrraaaayyyy directive or the ----llll option has    been used.

      Pattern-matching of NUL's is substantially slower than
      matching other characters.

      Dynamic resizing of the input    buffer is slow,    as it entails
      rescanning all the text matched so far by the    current
      (generally huge) token.

      Due to both buffering    of input and read-ahead, you cannot
      intermix calls to <stdio.h> routines,    such as, for example,
      ggggeeeettttcccchhhhaaaarrrr(((()))),,,, with _f_l_e_x rules and expect    it to work.  Call
      iiiinnnnppppuuuutttt(((()))) instead.

      The total table entries listed by the    ----vvvv flag    excludes the
      number of table entries needed to determine what rule    has
      been matched.     The number of entries is equal    to the number
      of DFA states    if the scanner does not    use RRRREEEEJJJJEEEECCCCTTTT,,,, and
      somewhat greater than    the number of states if    it does.

      RRRREEEEJJJJEEEECCCCTTTT cannot    be used    with the ----ffff or ----FFFF options.

      The _f_l_e_x internal algorithms need documentation.

     SSSSEEEEEEEE AAAALLLLSSSSOOOO
      lex(1), yacc(1), sed(1), awk(1).

      John Levine, Tony Mason, and Doug Brown, _L_e_x & _Y_a_c_c,
      O'Reilly and Associates.  Be sure to get the 2nd edition.

      M. E.    Lesk and E. Schmidt, _L_E_X - _L_e_x_i_c_a_l _A_n_a_l_y_z_e_r _G_e_n_e_r_a_t_o_r

      Alfred Aho, Ravi Sethi and Jeffrey Ullman, _C_o_m_p_i_l_e_r_s:
      _P_r_i_n_c_i_p_l_e_s, _T_e_c_h_n_i_q_u_e_s _a_n_d _T_o_o_l_s, Addison-Wesley (1986).
      Describes the    pattern-matching techniques used by _f_l_e_x



     Page 54                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      (deterministic finite    automata).

     AAAAUUUUTTTTHHHHOOOORRRR
      Vern Paxson, with the    help of    many ideas and much
      inspiration from Van Jacobson.  Original version by Jef
      Poskanzer.  The fast table representation is a partial
      implementation of a design done by Van Jacobson.  The
      implementation was done by Kevin Gong    and Vern Paxson.

      Thanks to the    many _f_l_e_x beta-testers,    feedbackers, and
      contributors,    especially Francois Pinard, Casey Leedom,
      Robert Abramovitz, Stan Adermann, Terry Allen, David
      Barker-Plummer, John Basrai, Neal Becker, Nelson H.F.    Beebe,
      [e-mail], Karl Berry, Peter A. Bigot, Simon Blanchard,
      Keith    Bostic,    Frederic Brehm,    Ian Brockbank, Kin Cho,    Nick
      Christopher, Brian Clapper, J.T. Conklin, Jason Coughlin,
      Bill Cox, Nick Cropper, Dave Curtis, Scott David Daniels,
      Chris    G. Demetriou, Theo Deraadt, Mike Donahue, Chuck
      Doucette, Tom    Epperly, Leo Eskin, Chris Faylor, Chris
      Flatters, Jon    Forrest, Jeffrey Friedl, Joe Gayda, Kaveh R.
      Ghazi, Wolfgang Glunz, Eric Goldman, Christopher M. Gould,
      Ulrich Grepel, Peer Griebel, Jan Hajic, Charles Hemphill,
      NORO Hideo, Jarkko Hietaniemi, Scott Hofmann,    Jeff Honig,
      Dana Hudes, Eric Hughes, John    Interrante, Ceriel Jacobs,
      Michal Jaegermann, Sakari Jalovaara, Jeffrey R. Jones, Henry
      Juengst, Klaus Kaempf, Jonathan I. Kamens, Terrence O    Kane,
      Amir Katz, [e-mail],    Kevin B. Kenny,    Steve Kirsch,
      Winfried Koenig, Marq    Kole, Ronald Lamprecht,    Greg Lee,
      Rohan    Lenard,    Craig Leres, John Levine, Steve    Liddle,    David
      Loffredo, Mike Long, Mohamed el Lozy,    Brian Madsen, Malte,
      Joe Marshall,    Bengt Martensson, Chris    Metcalf, Luke Mewburn,
      Jim Meyering,    R. Alexander Milowski, Erik Naggum, G.T.
      Nicol, Landon    Noll, James Nordby, Marc Nozell, Richard
      Ohnemus, Karsten Pahnke, Sven    Panne, Roland Pesch, Walter
      Pelissero, Gaumond Pierre, Esmond Pitt, Jef Poskanzer, Joe
      Rahmeh, Jarmo    Raiha, Frederic    Raimbault, Pat Rankin, Rick
      Richardson, Kevin Rodgers, Kai Uwe Rommel, Jim Roskind,
      Alberto Santini, Andreas Scherer, Darrell Schiebel, Raf
      Schietekat, Doug Schmidt, Philippe Schnoebelen, Andreas
      Schwab, Larry    Schwimmer, Alex    Siegel,    Eckehard Stolz,    Jan-
      Erik Strvmquist, Mike    Stump, Paul Stuart, Dave Tallman, Ian
      Lance    Taylor,    Chris Thewalt, Richard M. Timoney, Jodi    Tsai,
      Paul Tuinenga, Gary Weik, Frank Whaley, Gerhard Wilhelms,
      Kent Williams, Ken Yap, Ron Zellar, Nathan Zelle, David
      Zuhn,    and those whose    names have slipped my marginal mail-
      archiving skills but whose contributions are appreciated all
      the same.

      Thanks to Keith Bostic, Jon Forrest, Noah Friedman, John
      Gilmore, Craig Leres,    John Levine, Bob Mulcahy, G.T.    Nicol,
      Francois Pinard, Rich    Salz, and Richard Stallman for help
      with various distribution headaches.



     Page 55                         (printed 5/18/98)






     FFFFLLLLEEEEXXXX((((1111))))         VVVVeeeerrrrssssiiiioooonnnn 2222....5555 ((((AAAApppprrrriiiillll 1111999999995555))))           FFFFLLLLEEEEXXXX((((1111))))



      Thanks to Esmond Pitt    and Earle Horton for 8-bit character
      support; to Benson Margulies and Fred    Burke for C++ support;
      to Kent Williams and Tom Epperly for C++ class support; to
      Ove Ewerlid for support of NUL's; and    to Eric    Hughes for
      support of multiple buffers.

      This work was    primarily done when I was with the Real    Time
      Systems Group    at the Lawrence    Berkeley Laboratory in
      Berkeley, CA.     Many thanks to    all there for the support I
      received.

      Send comments    to [e-mail].











































     Page 56                         (printed 5/18/98)